A web-based database system for documenting x-ray crystallographic experiments.
The home page for Xtrack is at : http://xray.bmc.uu.se/xtrack
Xtrack keeps track of experimental data from protein expression through to crystallisation, data collection and processing.
The database is a glorified notebook system that allows different people involved in a crystallographic project to remember what they have done, and find out what other people have contributed.
The amount of data that could be tracked is essentially unlimited, and so to minimise the complexity of the interface, pointers to log files are used rather than having separate fields for more obscure properties. Exactly how many and which data fields to provide will no doubt be reconsidered and modified regularly. If you want to store information for which there is no specific field, use the 'notes' field.
The reference is Acta Cryst (2002). D58, 1889-1891
The system is based on information entered for a collection, an entity which corresponds to a set of X-ray data
collected for a particular structure determination. Each named collection
will belong to a particular project worked on by a group in a laboratory.
For example, a data collection for the pH-mutant cellulase work might be identified as : collection 5xph, and will belong to project CBHI, which is worked on by the group Cellulase in the laboratory Alwyn.
Note that in many cases a group will contain only one project in this terminology, in which case one can simply define a project with the same name as the group.
The lists of known laboratories, groups and projects will normally be maintained by the database administrator, and regular users will only add or modify collections and possibly projects.
Users : All users must be registered and assigned to one or more groups. The user may then read and write data only within those groups. Different passwords can be associated with the different groups or the user can choose to have the same password for a number of groups, in which case they will have access to all those groups when they log in with that password, but not to the groups associated with other passwords. Generic user accounts for use by several individuals can also be created.
A generic user account can be configured so that a stand-in user (which we call "Slave") can enter data for any group, but can only modify records that belong to Slave (ie the responsible field of the Structure page contains "Slave"). In this way a synchrotron visitor can collect and record data for other users, without the danger of accidentally modifying old data. True members of the project can then adopt the data entries by changing the responsible field of the Structure page, after which Slave will no longer have access to them (unless a group member chooses to set the responsible field back to "Slave").
There is a user account called 'Guest', which has no password and has limited read-only access to data. In the demo version of the program there is also a user called 'Demo' with pass word 'DemoPass', which can act as a normal user, creating and modifying records. Feel free to play in this demo version, but bear in mind that input data will not necessarily be kept.
Use : When the user logs in and comes to the front page of the database system, they are presented with the choice of searching for existing data or entering a new collection. These options are used as follows :
New Collection :
If they choose to add a new collection, the user is taken to a page which allows them to enter a collection name,
and to choose from a list of projects and groups. The database knows which laboratory the
groups belong to, so that need not be specified, but since in principle two groups could have projects
with the same name, the user has to specify both of these at present. This could change and be simplified if we decide that
all project names must be unique.
The user is also given the opportunity to enter free-text notes on the collection, and to name a responsible user. A priority can also be assigned, and if this is made numeric, it can be used
as a sort key for searches.
When they enter this data, a template is used to create default data associated with a collection, and the user is then free to enter the real data.
If no suitable project exists, all users have permission to create a new project under one of their groups. This must be done before trying to enter a new collection, since the name of the new project must be known when the new collection is made.
Data Entry :
Most items are entered as free text into the input boxes, but where there are a finite number of known responses,
a selection list may be offered alongside the free-text box. It is preferable to use the selection list if your choice
is included, as that will ensure consistent spelling and punctuation in the database.
Note that it is the value in the free-text box that is ultimately used, the option selector is only there to help you fill the box.
In free text, anything recognised as a URL (beginning with either http:// or file:/), will appear as a link on subsequent reading. If the URL points to medline or other recognised online libraries, the linked text will be contracted to "Web Reference" to avoid a big, ugly URL messing up your notes. If the URL is local (file:/), then it will only be valid when browsed from the local machine where the data resides.
Data entry is divided into pages, and links to the other pages that can be edited can be found at the bottom of the screen. The page currently displayed is greyed out, and pages that have not yet been edited are coloured mauve, but any page can be selected for further changes.
For extra information on what to put in the various fields, click on the field and help will appear in the status bar at the bottom of your browser (if you have one) until you move the mouse out of the box.
Sometimes selecting a particular option implies a particular entry in another option (for example selecting 'beamline = 711' implies that 'site = Lund'). If the program spots a dependency like this, then it fills in the default for next field for you.
On some pages you will be offered the option of extracting data from a log or coordinate file.
If you have entered the filename in question as an http address, then the program
immediately goes away and tries to get interesting data from it. Otherwise, you
will be offered a browse box with which to find your file.
For security reasons, this will happen even
if you have already entered a local filename. Annoying but unavoidable. The best you can do
is to cut and paste the displayed filename into the box.
If the data that is found in the chosen file makes some sort of sense, it will be used
to override corresponding fields in the page. If you do not enter a filename in the form box
before browsing for the file, the name of the browsed file will be entered for you.
Note that formats in log files change with time, and that this may cause Xtrack to extract bad numbers from
your particular file. If the number is absurd, you will be warned, but if it makes some sort of sense, it may slip by.
So always check that the extracted numbers are what you expect, especially after a program upgrade.
Files that Xtrack can currently extract useful information from include :
CNS, SCALEPACK and SCALA log files.
CNS, REFMAC and SHELX PDB files (and SHELX RES files).
If you have multiple datasets (either because you are collecting MAD data, or because you have separate
high and low resolution datasets), there are extra fields for entering collection data about those,
but they can be suppressed by selecting 'Native, single set' or 'SAD' in the 'Collection type' option list.
If you later change your mind, you need to set this to something else and hit submit
to refresh the page with the additional fields.
If you are collecting MIR data, then it is probably best to make a normal entry for the native collection, and then separate entries (with similar names) for the heavy atom derivatives, with a note on the collection page that subsequent processing is combined with the native set.
The notes field is for any extra information that you may want to add. The box may look small, but it grows each time it is saved with more text in it.
Some pages have fields for images, such as scans of gels. If a URL is entered into one of these fields, it will be replaced by a small in-line image linked to the full-sized image. In addition, you have the option to upload images to an archive server disk by using the "upload image" button, which will then append the archive URL to the contents of the 'image' field. A similar mechanism is available for uploading protocol files (eg Word or PowerPoint files).
If a search is chosen, the user can enter any information they know about the collection of interest, for example
the project or collection name, or can use the wildcard 'any'. When the search is issued, a list of all collections
matching the search criteria, and for which the user has read permission, is presented.
Leaving all the search options on 'any' will give a list of all collections that the user can see,
and that will normally be a manageable list.
If the keywords option is used, then entries will be selected if the strings entered in the
keyword box appear in selected fields of the relevant pages. Note that it does not search all the other pages
for the keywords, and that the keyword will also match substrings, so 'CBH-I' will match 'CBH-I' and also 'CBH-II'.
The conjunctions AND, OR and NOT can be used (the default being AND) as follows : "cbhI OR cbhII AND NOT alwyn"
Parentheses can be used to group the logic, for example : "( cbhI and ligand ) or cbh58".
The order of presentation can be controlled by the sorting options, and privileged users can choose 'report format', which shows the list in a format useful for making synchrotron-use reports.
Each entry in the resulting list has buttons with the options 'Examine', 'Edit', or 'Copy', and one of these options for the appropriate
collection is selected.
'Examine' presents all the data for that collection in a read-only fashion.
'Edit' presents the same data, with the option of changing values and saving the new data page by page.
'Copy' makes a copy of the data for that collection, and prompts for a new collection name. The default is to copy only selected data, only those fields that might remain unchanged for a new collection, but there is an option to copy all data fields, which can be used if you want to rename a collection (copy all, then delete the old collection).
Examine and Edit present one page of data at a time, along with a menu for moving to other pages.
There is also an option to list all pages one after the other, and to list a summary of those
statistics that are commonly included in a publication.
In addition there is an option to open a new browser window showing data on another collection. This allows easy comparisons of different sets of data, and a quick route to other collections.
Note that the search is only for collections, so if you haven't started a collection yet, your project won't show up. Use 'show projects in group' instead.
Note also that if you select a group or a project, the number of options in subordinate pull-downs get instantly reduced to only those that belong to that group or project.
On the Preferences page the user's password can be changed, the background colour can be customised, and a decision can be made as to whether the user's login name and password should be stored in a cookie. If you always work from a private workstation that can be useful, but if you use public terminals, you may not want password information to be stored. The password is encrypted in the cookie, but will allow anyone at that ip address to log in to Xtrack as you.
Crystallisation database :
From release v2.0 (spring 2003) Xtrack has the ability to store information about crystallisation experiments. (In previous versions you could only record a few details about the single crystal that was used for a particular collection).
Because they don't map easily onto collections, crystallisation trials are considered to belong to a project, and can be found by doing a "list projects in group" on the main Xtrack page, and selecting "xtal" for the project of choice. Alternatively you can choose the "Crystallisation" button on the front page, which will take you to a facility for searching for plates of interest, for example all those with crystals with scores above a certain threshold.
After following the project link or performing the search, you are presented with a list of known crystallisation plates, from which you can make a selection. Alternatively you have the option to make a new plate with a name, size and number of drops_per_well of your choice. The default name that you are offered is formed by guessing what your most recent plate is, looking for a number at the end of that plate's name, and incrementing that number. If no number is found, a '2' is added to the name; if you have no plates, the default name is 'p1'; and if you have an irregular naming system no default is offered and you must type something in. The plate name must be unique within the project, but since it is associated with a project you do not need to include the project name in the plate name, and it is probably better not to, in order to keep the name short.
Plate editing :
Once you have chosen an old plate or created a new one, you are taken to the plate editing page. Here you are presented with a table
of plate properties (some of which you can change), a bunch of controls for changing the contents of the wells,
and a table representing the plate with all its wells. There is a 'save' button that must be used if you change the header
information of the plate, and an 'update page' button to implement option changes.
Note that the header information will not be updated if you use the well-adding controls or 'update page' before
doing a 'save'. On most browsers 'save' will flash annoyingly to remind you of this if you make a change, and if you still try,
you will be asked to confirm that you want to ignore any changes.
The next decision to make on this page is how much of the information stored for each well that you want to be presented on the page. This will depend on the size of the plate, the size of your computer monitor, and at what stage of the experiment you are. The choices include only displaying the recipe that you need for making the plates (to print out and take to the lab), only displaying the score (when you are recording what you see when you examine the tray), or showing everything. There are other combinations too, like 'results', which means 'score + comments', which is another option you might want when examining the tray. You can also choose whether you want to be able to 'Setup' the well conditions, only 'Show' the data, manually 'Edit' the well conditions, show the data in a 'Printable' form (ie with inessential information left off the page), or output a control file for a crystallisation 'Robot'. Then you choose the size of the font to use, and whether to work with all wells, just an individual well, or all wells with a score above a certain value.
If you choose the robot option you will always get output for all the wells, and it will be formated as a page of key/value pairs that can be easily interpreted by a filtering program that can in turn output commands in a format appropriate for your specific robot. At the moment only one dummy filter is provided to demonstrate how you would write your own.
Adding components : This is where you specify what is in your reservoir solution and drops, in terms of concentration of protein and buffer components, and may involve using a standard screen or making customised optimisation plates.
To set up a plate with a commercial crystal screen, simply choose the name of the screen from the pull-down menu of abbreviations,
and if your plate is smaller than the size of the screen, choose which solutions to put on the plate by specifying
the start and end numbers. The interface tries to be clever here, so you only need to enter the start number, and it will work out
the last one. Also, if you make a new plate immediately after adding part of a screen, the defaults should increment so that
you are ready to add the next part of the screen to the new plate. Changing the screen name resets the starting well to 1.
But what if you want to skip solutions 25 and 27 from the Hampton screen ? Then you use the 'add screen' button several times, changing the 'Well start' field to start adding from the well of your choice each time. But it's probably easier to skip tubes 49 and 50 instead.
After adding the screen, the table representing the plate will be filled in with symbolic names for the screen solutions, or if you are lucky, it will run off to the XtalScreens server in Uppsala, and look up the recipe for that particular solution, and put that in too (which incidentally you can run by hand at http://xray.bmc.uu.se/markh/php/xtalscreens.php).
Note that when adding a screen to the reservoir, the previous contents of the reservoir are erased, so if you want to mix your own solutions with a screen, you need to add the screen first, and then your solutions. Adding screens to the drop does not erase previous contents, since you may want multiple additives.
If you don't recognise the abbreviations used for the screens, select 'help' from the bottom of the pull-down menu and you will get a page with the full names and other data about the screens.
Most people use their screen solutions across the plate, but you may want to run them down instead (for example the Molecular Dimensions Footprint screens have 24 solutions arranged as adjacent quadruplets). In that case you select 'Down' in the order menu.
Optimisation plates : To set up an optimisation plate there is an input box where you can specify what stock solution you want to add to the reservoir by choosing from the pull-down menu of abbreviations. When you choose an item here, you will be reminded of the composition of that stock by the full description being shown in your browser's status bar (if you have one). At the same time, the 'Units' field will be filled in with the appropriate units stored for that stock. Note that you must work with those units, and in most browsers that will be enforced by disabling the 'Units' menu for user selection. If the stock that you want is not in the menu, you will need to edit your stock list, as descriped later on this page.
Now there are three different ways to specify how you use that component. In the bright yellow
boxes you can specify the starting stock volume, and how much to increment the volume each time you move down a row or across a column.
In the bright green boxes you can specify the starting concentration, and how much to increment the concentration between each row/column,
and in the olive-coloured boxes you can specify the starting pH, and how much to increment the pH between each row/column.
(Some users have asked for the option to give starting and ending concentrations, rather than starting and incremental concentrations, but this would most likely result in recipes requiring irregular volumes that would be awkward to dispense, so I have chosen to protect them from themselves.)
You must also specify whether to increment the concentration down the rows of the plate, along the columns, or not at all, and you can choose to work with a subset of all the rows and columns.
When you hit the "add stock" button, the wells will be updated so that the 'composition' field will show the current
concentration of components (using their abbreviated names), and the 'recipe' field will show how much of each stock you must add,
along with a calculation of how much water needs to be added to fill the well.
You can then add as many components as you like, and if you want to add one to all wells equally, simply set "inc vol' to zero (which will be done for you if you choose 'No increment').
You will be warned if you add more solutions than the well can hold.
If you don't want to fill the plate, or you want to add something special to individual wells, you can choose the start and end rows/columns in which to make the additions.
If you want to increment the pH of a single buffer across the plate, use the 'start pH' and 'end pH' fields, and choose the stock that is your
variable buffer. 'inc vol' will automatically be set to zero when you do this. If, on the other hand, you want to increment the pH across
a plate using different buffers, then you must enter each row/column separately giving the name of the new buffer each time.
The interface will try to help you by incrementing the start and end row/column values each time you add to a single row/column.
If you add to just a single well, the well is not incremented by default after adding, but if you click in the 'Start col' box, the columns will autoincrement once.
Once your reservoir is set up, you use another input box to assemble your drop. You can add protein and reservoir solution separately, or you can use "Prot and Res" to add both in one step (if you choose 2ul as the volume, 2ul of each will be added). You can also add metals, ligands, detergents etc.
At the bottom of the page, a list of stocks used will shown, including the abbreviation and the full description of the stock.
This system should allow you to set up an arbitarily complex protocol, but if you really need to note something in a well that cannot be done with these input controls, you can always edit the well manually. Note however that manual editing can lead to inconsistencies between, for example, recipes and compositions.
If you make a mistake while setting up the reservoirs, or if you wish to edit an existing plate, there exists a limited editing function in the 'remove stock' option. This searches the reservoir and recipe contents for occurances of the selected stock, and removes any lines referring to that stock, recalculating the water content as it does so.
New plates : If you are setting up a series of plates, there is a "Make similar plate" option that creates a new plate with similar plate conditions, but no well information. But if you want to make a completely new plate (a different size, for example), you need to go back to the 'Plate list' page where more options are avaialble. If you have a well protocol that you like, you can use it again later by creating a new plate and doing a 'Copy Wells' from the old plate.
Editing the stock list :
The stock solutions that appear in the pull-down menu are maintained by pressing the
'edit stock list' button (or choosing the 'edit stocks' option at the bottom of the stock list menu),
which will open a new window and give you a list of known stocks, with the option to add new ones. Each stock is given a short name that is used
in the menu, a full description of what is in the stock, the concentration, a batch number, and a type flag which tells
whether the solution is a buffer, a ligand etc. The 'type' of the stock determines which menus it will appear in,
(buffers and precipitants are not offered for adding to just the drop, for example),
and may be also used in future functions, but do not lose too much sleep over deciding what type best describes your stock,
and feel free to leave it blank.
It is recommended that the abbreviation chosen for buffers of fixed pH includes the pH in the name, eg. Tris8.8,
but variable buffers be identified by just a name, as the pH will be listed alongside when it is used.
For a stock solution that has several components, the concentration should be given as "100%",
so that when it is used later, its contribution to the resulting solution can be expressed as a percentage of the
stock concentration. Otherwise the concentration should be given in the units that will be used in the well,
for example 1M Cacodylate should probably be entered as 1000mM, otherwise you will have to express
the concentration required in the well as 0.05M instead of 50mM.
At the bottom of the page there is also an option to import the stock list from another project in the groups of which you are a member. All the items in the other stock list that don't have the same name will be added to your list. If you are using a robot in your lab, you might want to create a dummy project called 'robot' from which users can import the standard robot stocks.
After adding stocks with this utility you must do an 'Update page' in the plate editing window in order to see the new stocks in the drop-down lists. When you are finished with the stock-editing window, there is a button at the bottom which should make it disappear, but may not.
Entering results : There are two ways to enter the results of your crystallisation trial, interactively, or by feeding in a file of results. The latter is meant for robots, but can also be used in a normal lab if for example you have no ethernet connection in your cold room, and enter your results on a laptop in there, transfering the data later.
To enter data interactively, choose 'Edit', 'Results' and 'All Wells' from the 'update page' menus.
You will then be presented with fields for entering score and comments for each well. You can put any text in these
boxes, but the idea is to use 'score' to keep a single number representing the final quality of any crystals in the well,
and 'comments' to keep a running commentary on what is happening in the well over time. Experienced users can
quickly jump from score to score while looking down the miscroscope, and as they enter the standard single-digit codes (defined in
the database at each installation site), more descriptive text defined for that code and the day's date will be automatically appended to the comment field.
Less experienced users may prefer to pick from the menu of descriptions, which will cause that description to be appended
to the comment box, and the corresponding score to overwrite any previous entry in the score field (even though it may not be immediately visible if you are using Explorer).
If you wish to add other text, or erase automatic entries, you can easily do so.
Note that selecting 'Edit' and 'Results' is a special case with respect to formatting, in that the score and comment entry boxes are displayed separately, in two different tables. The idea is that it is easier to tab from well to well as you type in scores, but you still get the expanded descriptions further down the page. All other combinations of options squish all information about a well into the same box in a single table.
And note also that if you edit only 'Score', then the comments fields will not be updated with the expanded interpretation of the score number, so you would normally not want to use that combination.
You will also sometimes see a line called "Fast entry". This is really there to support voice recognition, but can also be used for fast keyboard entry. You enter the row, column (drop) and score as single characters into the boxes, and the appropriate fields are filled in. Focus is automatically shifted to the next box as you type (or speak) a character, so there is no need to touch the mouse or hit carriage return. Focus shifts back to 'Row' each time you enter the score. One drawback with this entry method is that if you have a 96-well plate, then you must refer to columns 10, 11 and 12 in hexadecimal, ie as columns A, B and C.
There is an option to upload crystal photos to a central archive, but it is only available when editing a single well with the photo field included in the options. If your local installation allows it, selected crystallisation data is embedded into the EXIF header of the image file, so that its origins can be traced later, even if it becomes separated from Xtrack.
To use the "Import results" button, create a file with one of the following two formats. Either a simple list of well/score items:
# Simple format
A3-2 4 # (This means drop 2 in well A3)
or a list of "item = data;" records in the following format :
# Advanced format
Row=A; Col = 1; Result_score = 1; Result_comment = Plates dissolved again. Bugger;
Row = A; Col = 2; Drop = 1; Result_score = 6; Photo = http://xray.bmc.uu.se/markh/php/crystal.jpeg
Project = CBH-II; Plate = p107; Row= B; Col = 1; Drop = 2; Result_score = 3 # (If you specify a drop number, it must be appear before the score)
You will then be invited to upload your data file, and will be presented with Xtrack's interpretation of your file. If you are happy with that interpretation, hit "confirm" and the results will be entered into the database for your current plate. Scores will overwrite current data and append the text equivalent to the comments field, while photos and explicit comments will be appended to the old ones along with the current date.
The keywords 'Project' and 'Plate' are redundant for non-privileged users, as they can only import results for their current plate, but privileged users can import for any plate in any project. This is of course a very dangerous command, since a typo resulting in inconsistency between project and plate name could result in overwriting data in arbitary plates. The reason for including them is for use by robots, which can hopefully be trusted to write out large amounts of consistent data, which it would be inconvenient to have to break up and feed in separately for each plate. You will need to write your own filter to convert the natural output of your robot into Xtrack's input format.
Incidentally, you can also use this input method to populate the content and recipe fields, if you have designed a screening scheme that is so irregular that it is a nuisance to input through the interface. Use a syntax of the following form :
Project = CBH-II; Plate = p107; Row= B; Col = 1; Components_final = "15% PEG 400\n0.1M Na Acetate pH4.6"; Components_add = "JBS1-1"; water_vol = "500 ul Water";
Voice recognition on Mac OSX :
VoiceCommander (http://www.ex-cinder.com/voicecommander.html) is a very simple and cheap ($15) utility for that allows for vocal input of results on Mac OSX while you are looking down the microscope. By saying commands of the type "clear next cloudy next plates next next next needles" you can quickly fill in sequential data without touching the mouse or looking at the screen. Alternatively you can use 'fast' entry if most of your wells are empty. Then you say something like "A 4 cloudy B 3 plates C 2 spherulites". The voice recogniser is very easy to install and setup, and if you pick up the templates from http://xray.bmc.uu.se/markh/php/export/xtrack_voice.tar you can do it in 5 minutes. Choose the appropriate voice template ('seq' for sequential entry, 'fast_24' or 'fast_96' for the fast-entry box), then select 'Edit' 'Result' in Xtrack, click on the first score box or 'fast entry', and start talking. Possible commands are listed in the VoiceCommander window, and should be self-explanatory. The 'all' template could be used for any input, but contains so many possible words that misunderstanding is more likely, so it's better to use the more limited templates. The facility is still under evaluation in v2.1, and better documentation will follow if it works out.
Connecting to the rest of the Xtrack database :
When you have a successful crystal on which you collect data, you will want to link that to your entry for the data collection. You do this by editing the crystallisation page of that entry, and choosing 'Extract data from crystallisation database'. You will then be prompted for the plate and well name (if it wasn't already given in the appropriate field of the page), and you will be shown the data for that well. You are then prompted for confirmation that the data is correct, after which it will be entered in the main data-collection database. At the same time, the name of that collection will be added to the "Usage" field of the plate's well in the crystallisation database, and that will be displayed as a button that allows you to click back from the crystallisation page to the Xtrack collection page. After this you will also find a short cut appears at the bottom of the Crystallisation page that will take you directly to the xtal db entry for the appropriate plate.
Apparently I'm legally obliged to tell you that Xtrack sets cookies on your computer if you ask it to save your preferences. The information saved is your encrypted Xtrack username and password, and your chosen background, and I promise that I don't sell this information to anyone else, however much they offer me to know what your favorite colour is. You can prevent any cookies from being set by choosing the appropriate option in your browser's preferences. There, that should keep me out of jail.
Known bugs :
1) Using the Konquerer browser, input checking fails, so it is possible for instance to enter a
badly-formated date for the collection.
Workaround - use a decent browser.
2) Sometimes the option box for selecting wells shows only "all wells".
Workaround - Click "Update page" to get the individual wells added to the option list.
(But I think this is fixed now.)
3) Some SCALEPACK log files are too big for some browsers to upload, and Xtrack
fails to read the file and simply returns to the upload page.
Workaround - truncate the file and upload only the relevant summary lines at the end.
(Also fixed for the biggest files yet produced.)
Administrators help file