StoreMan
StoreMan is a Linux GUI - program, written in Python and based on the wxWidgets cross-platform Gui-library. The software is a combination of a comfortable backup program and a db based archive manager.
StoreMan is ideal suited to store backup / archive files on a (Free)NAS box and on
USB disk drives (as well as on a local harddisks) - and for searching files without the
need to have the storage devices permanently online.
Contents
To run StoreMan, you need Python >= 2.5.x (not 3.x) and wxPython 2.8.x.x or better installed on your computer. In addition it would be wise to install the following module:
StoreMan has been tested on computers running Kubuntu Linux (KDE), Ubuntu Linux (Gnome) and Windows XP SP2 and SP3.
Introduction
At the very first start of StoreMan, two files will be created in your
home directory:
Later you can decide to store these files elsewhere on your computer.
StoreMan works with tasks and jobs. In short, a task holds the
information about what to do and the job holds the information about
when to do it. A job without an associated task is called useless.
Normally one or more tasks are associated with a job. A task can be
associated with more than one job. A job can be executed and runs the
task(s) associated with it. A task cannot be run without a job.
If you have already backup files stored on your storage device(s) it
would be a good idea to scan these devices first. The scan will collect
all relevant information (dir, filename, size, ...) and store these in
the database. To start the Scan manager, select Scan - Manage scan dirs
or type CTRL-S or use the context menu.
Secondly you may want to create one or more tasks. You do this
by
calling the Task Manager. This can be done by clicking Backup - Manage
task(s) or by typing CTRL-T or via the context menu - click the right
mouse button and choose Manage task(s).
In the Task Manager you click New to create a new task. A New / Edit
backup task dialog will pop up. You can change the default taskname,
set the source and destination paths for the backup, decide if
subdirectories should be included in the backup or not. You can edit regular
expressions to exclude files and / or directories from the backup and
you can select the compression method which has to be used.
Thirdly you may want to create one or more jobs. You do this
by calling the Jobs Manager. This can be done by clicking Backup -
Manage job(s) or by typing CTRL-J or via the context menu - click the
right mouse button and choose Manage job(s).
In the Job Manager you click New to create a new job. A New / Edit
backup job dialog will pop up. You can change the default jobname,
decide to mark the job for immediate execution or for scheduled
execution (or both), select date, time and interval for scheduled
execution and - most important - you can select, which of your
previously defined tasks should be associated with this job.
Now you are ready for a first backup run. If you have marked one or
more
jobs for immediate execution, these jobs are highlighted in the
Backup jobs sub-window. If you have not chosen one, you can do so by
clicking on the
job with the middle mouse button / wheel. After that, click on the
Immediate
Job Execution shortcut icon or menu or context menu. Now you are
set -
your job will be executed.
As a last step you may want to execute a query on the database (now holding
the information from the previously executed device scan and from the job
execution). You can press the shortcut Execute query icon or use the menu
or context menu. After a short delay (depends on the speed of your computer and
the number of entries in the database) you will see all information in the query
results window. You can now refine your query by using the filters.
StoreMan GUI has a main window for database Query results and 4
small sub-windows named Backup jobs, Scan dirs, Databases
and
Log messages. You can close and
reopen these small windows (using the View menu) or you can drag
these windows around and
place them at different positions either inside or outside the main frame.
The main window includes a menu- and toolbar with shortcut icons for quick access of the most wanted actions. It also includes filter comboboxes with histories. Query filters are available to narrow down directory, file, size and / or date.
The % sign is the wildcard char in the filter fields. This char is
placeholder
for 0 to n chars. For example: %jpg matches 'jpg', 'picture.jpg' but
not 'abc.jpeg'. You can use 0, 1 or more wildcards in one field. %wil%
matches 'wildcard', 'wil', 'in_th_wild' and so on.
If you mark a line in the Query results window by clicking it, the directory part of that line is copied to the system clipboard. You may want to paste it into another application (i.e. a text editor).
If a line is marked, you are also able to delete the item from the storage device and from the database. This can be done via the File menu, by typing CTRL-R or via the context menu entry.
If you double click a line in the Query results window, the
preselected filemanger (see Program
settings - Listcontrol) will start and open the directory
which is the
first part
of the line.
And finally, the main window contains a statusbar divided into 4
fields. From left to right they are:
The status led is implemented to help you to know, what's going on with the app (see table).
led color | meaning |
---|---|
red | no connection to the selected database could be established |
blue | scan is in progress |
light blue | backup / copy job is being executed |
yellow | database query in progress |
light red | not all available matches are listed in the Query results window (see 'Preferences', 'Display limit') |
green | ready - waiting for actions |
The first of the two multi function fields (field #3 and field #4) can indicate backup statistics (B:), matchcount (M:) or recordcount (R:).
See examples in table below.
field #3 | field #4 | context |
---|---|---|
B: 1.207, 2, 1 | /data/prog/python/StoreMan | backup task running, 1207 files read, 2 files skipped, 1 error (i.e. unable to open) |
M: 127 | /opt/StoreMan/storeman.db3 | database query executed, 127 matching results |
R: 14.210 | /mnt/mpmd/home/mersmann | directory scan executing, 14210 entries are currently in the database. |
Common program setup is done via the Preferences menu. To get to the
menu, use
'Options', 'Preferences', type CTRL-F or use the context menu (right
mouse click).
The Preferences notebook dialog has 4 tabs.
Misc:
Note for Windows users: the StoreMan application does not put any values in the Windows registry.
Logfile remark:
Logfiles are written by a RotatingFileHandler with a maxBites size of 64 / 512
kB and a backupCount of 4 backups. Rollover occurs whenever the current
log file is nearly maxBytes in length. If backupCount is >= 1, the
system will successively create new files with the same pathname as the
base file, but with extensions ".1", ".2" etc. appended to it. For
example, with a backupCount of 4 and a base file name of "app.log", you
would get "app.log", "app.log.1", "app.log.2", ... through to
"app.log.4". The file being written to is always "app.log" - when it
gets filled up, it is closed and renamed to "app.log.1", and if files
"app.log.1", "app.log.2" etc. exist, then they are renamed to
"app.log.2", "app.log.3" etc. respectively. This is the same technique
as you might have seen in the /var/log dir of Linux systems.
The format of a logfile line is: date time [- taskname] -- message i.e.
Action-log (64 kB)
2010-02-09 09:42:49 -- DB-connect - database '/opt/StoreMan/test.db3' opened
Backup-log (512 kB)
2010-02-09 09:43:33 - weekly -- Skipped (exclude match): /home/mersmann/.bazaar/explorer/bookmarks.xml~
Mount fs (feature only on Linux systems available)
The Manage tasks dialog is shown below.
In this dialog you see the existing tasks in the left pane and all
information about the selected (highlighted) task in the right pane.
In the main window (Backup jobs sub-window part), you can gain access to quick information about a
task after expanding the job the task belongs to and by right clicking the task item. You can navigate in the Backup
jobs sub-window with the mouse, by typing the first char of a job- / taskname
and by typing the '+' (expand all jobs) or the '-' (collapse all jobs)
sign.
The New / Edit tasks dialog notebook has 5 tabs.
If the New / Edit backup task is started with New, you can change
the taskname in the Task Id tab from the default
task-current_date_and_time to a name that fits your needs. If the start
is via Edit, the taskname is not changeable.
In the Paths tab you can
In the Exclude re's tab you can define which directories and / or
files should not be included in the backup. This is done via
regular expressions. If you press Add, Add dir or Edit and insert or edit a
regular expression and hit RETURN,
the expression is compiled and validated. If the expression is valid,
the status column indicates 'valid', if not, the status column shows the
word 'error' and the expression is shown in red.
In the upper part of the dialog files and dirs from the Source path (tab Paths)
are listed (wihout the contents of subdirs) as an example. The valid regular
expressions are applied and matching lines are coloured. So you can
test if the defined regular expressions fit your needs.
If you press the help button you will see a html page from the python
reference library with an explanation of the re grammar.
Remark: re's are always applied to the full expanded filenames (fefn).
Examples:
You want to save your /etc directory but do not want to save editor backup files with
the format 'name~' in your /etc dir, nor do you want to save the subdir
apt (files and subdirs) and last but not least you do not want to backup the dirs /etc/rc0.d,
/etc/rc1.d and so on.
The full expanded filenames (fefn) in this case are for example '/etc/fstab',
'/etc/vsftpd.user_list', '/etc/apt/sources.list' and so on.
You need three regular expressions to accomplish the mission:
Regular expr. (re) | Explanation | Matching fefn |
---|---|---|
(.*)~$ or /etc/(.*)~$ |
The first '(.*)' says - skip all fefn starting with any char (the .), repeated 0..n times (the *). '~' says - the next char has to be a '~'. '$' says - now the line must end. | /etc/fstab~ |
(.*)/apt/(.*) or /etc/apt/(.*) |
The first '(.*)' says - the fefn to be skipped has to start with any chars followed by '/apt/. The last '(.*)' says - skip any following chars. | /etc/apt/sources.list |
/etc/rc.\.d(.*) or (.*)/rc.\.d |
In this example the matching fefn has to start with '/etc/rc'. The '.' says - now skip one char. To match a '.' in a file or dir name you have to escape the '.' because of its special re meaning. Escaping is done by the '\' - so we have to type '\.'. The next char has to be a 'd'. A third alternative is the re '/etc/rc.\.d/(.*)' and there are more. | /etc/rc5.d/README |
On the Compression tab you decide either to use (create archive) or not to use compression (only copy
files). If you want to use
compression, you have the choice of several methods. In the latter case
the next tab is activated.
In the Misc tab you can choose if your compressed backups (archives) are to be
prepended or appended by a date - time - stamp. This is very useful,
because if you decide to use the stamp, there is no danger for
overwriting an older file - except you run the same job twice in the
same minute.
In this dialog you see the existing jobs in the left pane and all
information about the selected (highlighted) job in the right pane.
In the main window you can gain access to quick information about a job by right clicking the item in the Backup jobs sub-window. You can navigate in the Backup jobs sub-window with the mouse, by typing the first char of a job- / taskname and by typing the '+' (expand all jobs) or the '-' (collapse all jobs) sign.
The New / Edit jobs dialog notebook has 3
tabs.
In the Job execution tab you can
The Manage scan dialog is shown below.
In this dialog you see the directory tree of your system in the left
pane, a history of previously selected dirs in the upper right pane (for quick navigation) and
the currently selected scandirs in the lower right pane.
The Scan dirs sub-window shows in the upper part the dirs currently
selected for a scan run. This information is saved in the INI file.
In the lower part the window lists the dirs that are already scanned. This information is stored in the currently used database.
If you execute the defined scan ('Scan', 'Execute scan' or type
SHIFT-CTRL-S), all directories are scanned
recursively and the
filenames (with dir info), sizes and creation dates are stored in the
database for queries.
Important: You should execute a scan on a non empty storage
devices, before executing a backup job. So the database will be in sync
with your storage device.
StoreMan uses a built-in standard Sqlite database to store backup
information. You can inspect the interior of the database (if you like) with any Sqlite database
tool, i.e Sqliteman (http://sqliteman.com/).
Database handling is done byYou can select a database via the Database menu, by typing CTRL-O or by
double clicking on an item in the Databases sub-window. Database(s) store information
about scanned devices and backup runs - NOT the backup data.
The Manage database dialog is shown below.
The left pane lists the Known databases. The right pane shows information about the selected database.
In the main window, you can gain access to quick information about a
database by right clicking the item in the Databases sub-window.
All screenshots shown in this document, are taken from StoreMan 1.2x or 1.3x running on Linux (Kubuntu 10.10, KDE 4.x).
If you like the software, use it (at your own risk). If you have
suggestions or encounter bugs - please send me an email. If you do not
like the software - throw it away.
![]() |
![]() |
StoreMan was designed and coded by Frank Mersmann
(frank.mersmann@gmail.com).
Beta tests where conducted and hints were made by Wolfgang Bredow
(bredow.w@googlemail.com).
1.34.20110218