Reconstructing LXR Databases


  Flag for language en   Flag for language fr 
You went through a complete system upgrade and, even if you were cautious enough to save your private data (/home and some /usr directories), you discover all the databases have disappeared because the servers store them in obscure locations which were blown away with the system upgrade. Installing a newer LXR release may also result in the need to fully reconfigure subsequent to lxr.conf loss. It becomes really tedious when you have more than 10 trees. The task is error-prone, time-consuming and brings no added value. Fortunately, the process can be automated.

Script recreatedb.pl ships with LXR 1.0 and higher.

Required backup before upgrade

The bare minimum needed to reconstruct the LXR server is the master configuration file lxr.conf.

To mimimize the risk of inconsistency, save also the context file lxr.ctxt located in the custom.d/ directory.

Ideally, fully backup the custom.d/ directory, principally if you indexed a kernel tree and used script kernel-vars-grab.sh to collect the architecture names. But the collection step can be done very quickly if you did not backup.

After upgrade

The LXR configuration and database creation steps are replaced by the following procedure.

  1. Reload the saved files

    As mentioned above, you need at least the master configuration file lxr.conf and the context file lxr.ctxt. Put them into the custom.d/ directory.

    If you forgot to backup lxr.ctxt, it can be rebuilt from specific questions. But, BEWARE, if your answers do not match the initial context, you may end up with an inoperative LXR server. In that case, restart the procedure here with different answers.
  2. Run script recreatedb.pl IMPORTANT! This script is designed to be run from the LXR root directory. Eventually type command: $ cd /LXR/root/directory $ ./scripts/recreatedb.pl --verbose custom.d/lxr.conf *** LXR DB initialisation reconstruction (version: 1.1) *** LXR root directory is /home/myself/lxr Configuration read from custom.d/lxr.conf Initial context custom.d/lxr.ctxt is reloaded Your DB engine was: MySQL Configuration file custom.d/lxr.conf loaded *** scanning global configuration section *** *** scanning /lxr/tree1 tree configuration section *** *** scanning /lxr/tree2 tree configuration section *** *** scanning /lxr/tree3 tree configuration section ***

    recreatedb.pl creates a script custom.d/initdb.sh containing SQL instructions to rebuild all databases corresponding to the trees described in the master configuration file.

  3. Recreate the databases $ ./custom.d/initdb.sh See the notes related to this script in the configuration step .
  4. Eventually, collect kernel architectures

    Should one of your trees be the Linux kernel, launch kernel architecture enumeration with this script.

    This step is optional if you backed up custom.d/ and you did not change the kernel source tree.
  5. Populate the databases $ ./genxref --allurls This is the only case where option --allurls is recommended. Under some unclear circumstances, reindexing PostgreSQL trees may fail. In that case, abort the running genxref with control-C and relaunch individual genxrefs from the failing tree onward: $ ./genxref --url=... --reindexall Option --reindexall is only necessary on the failing tree to force a clean database.

Web server configuration is done as usual. See here.

Advanced usage

Non-default operation

Use --help option and experiment.

Incomplete backup

Incomplete backup here means lxr.ctxt has not been saved.

Script recreatedb.pl tries to restore a missing context (but does not save it to avoid creating a permanent inconsistency). This is done through a set of questions:

$ ./scripts/recreatedb.pl --verbose custom.d/lxr.conf ERROR: configuration context file custom.d/lxr.ctxt does not exist! *** LXR DB initialisation reconstruction (version: 1.1) *** LXR root directory is /home/myself/lxr Configuration read from lxr.conf WARNING: could not reload context file custom.d/lxr.ctxt! You may have deleted the context file or you moved the configuration file out of the custom.d user-configuration directory without the context companion file custom.d/lxr.ctxt. You can now 'quit' to think about the situation or try to restore the parameters by answering the following questions (some clues can be gathered from reading configuration file lxr.conf). WARNING: inconsistent answers can lead to LXR malfunction. Do you want to quit or manually restore context? [QUIT/restore] >

Choose the default Q answer to eventually add an --lxr-ctx= option to designate an existing context file in an unusual location. Answer R to enter the following interactive dialog:

The following questions are intended to rebuild the global databases options (which may be overridden in individual trees). Answer with the choices you made previously, otherwise your DB will not be what LXR expects. Default database engine? [MYSQL/oracle/postgres/sqlite] > Configured for single/multiple trees? [s/M] > The safest option is to create one database per tree. You can however create a single database for all your trees with a specific set of tables for each tree (though this is not recommended). How did you setup the databases? [PER TREE/global] > All databases can be accessed with the same username and can also be described under the same names. Have you shared database characteristics? [YES/no] > Did you use the same username and password for all DBs? [YES/no] > --- DB user name? [lxr] > --- DB password ? [lxrpw] > Did you give the same prefix to all tables? [YES/no] > --- Common table prefix? [lxr_] >

After this dialog, recreatedb.pl proceeds as previously.

Note that default operation is here multiple because trying to reconstruct the databases makes sense only when there are many. If you manage only one tree, it is much safer to reconfigure with the standard installation procedure.