Introduction to the Standalone WWW Blast server

Index

Introduction

This standalone WWW BLAST server suite of programs was designed similar to the regular NCBI BLAST server and such command-line NCBI BLAST programs like "blastall", "blastpgp", "rpsblast" and "megablast". It incorporates most features, which exist in NCBI BLAST programs and should be relatively easy to use. This server does not support any request queuing and load balancing. As soon as the user hits a "Search" button, BLAST starts immediately if entered information is valid. So, this server is not intended to handle large load, which may exist in public service. Such queueing and loadbalancing however may be implemented using such products as Load Sharing Facility - "LSF" from Platform Computing Corporation. Interface to "LSF" was implemented in NCBI, however this was not included in this suite. Standalone server assumes that users have their own BLAST or RPS-BLAST database(s), that should be searched and want to have a simple WWW interface to such search. It is STRONLY recommended that user have experience in installation and running standalone NCBI BLAST programs.

After files are uncompressed, server is ready to be used immediately. Any customizations to the program are welcomed. The source code for WWW BLAST is now part of the NCBI C toolkit, which can be downloaded from the NCBI FTP web site: ftp://ftp.ncbi.nih.gov
The compilation of the NCBI C toolkit includes the WWW BLAST executables. The files in the C toolkit most relevant for the WWW BLAST executables are: wwwblast.c, wwwbutl.c, psiblast.c, wblast2.c.

What's new in this revision?

May 10, 2004
  • XML output brought in sync with the text output, masking filtered locations in query sequences.
February 2, 2004
  • All source code moved to the NCBI C toolkit.
  • Mouseover feature fixed in graphical overview.
  • All binaries synchronized with the latest NCBI C toolkit release. All future releases will be automatically synchronized with future toolkit releases.
  • Recompilation procedure changed. WWW BLAST binaries are now compiled as part of the C toolkit.
November 21, 2003
  • Fixed a bug with absent images on PSI-BLAST iterations.
  • Removed header and progress messages from XML output.
  • All binaries recompiled with the newest version of the NCBI C toolkit libra ries.
May 9, 2003
  • Cleaned the Makefile for recompiling the sources.
  • Added discontiguous Mega BLAST options.
  • Added option to choose a subsequence location for query.
  • Added option to choose database genetic code.
  • All binaries recompiled.
April 21, 2003
  • Added instructions for recompilation of binaries from the source code to this readme file.
  • Fixed a bug in processing of the configuration file that limited allowed number of databases.
  • Added images for the Linkout and UniGene links from the BLAST output.
  • Corrected paths to the graphical overview for PSI/PHI BLAST output.
  • All binaries recompiled with the newest version of the NCBI C toolkit libraries.
February 10, 2003
  • Added support for multi-query XML output.
  • All binaries recompiled.
December 2, 2002
  • All binaries recompiled with the newest version of the toolkit.
August 6, 2002
  • Recompiled all binaries to include recent bug fixes.
June 18, 2002
  • Removed limit on the total length of database names.
  • Removed xmlblast.cgi and other unneeded XML related files.
  • Recompiled all programs.
January 11, 2002
  • Removed formatdb, makemat and copymat binaries from the archive - those should be taken from the NCBI BLAST executables archive
  • Added tabular output format
  • Added option to mask lower case in query
May, 2 2001
  • No major changes. All programs have been recompiled and made synchronous to the latest NCBI tookit release.
November 3, 2000
October, 19 2000
September, 28 2000
  • Added possibility to limit search to results of Entrez query (Regular client/server BLAST)
September, 11 2000
  • Added MEGABLAST search.
  • Added possibility to have multiple FASTA query input - batch searches with multiple graphical overviews. (Regular BLAST)
August, 22 2000
  • Added new advanced statistics to PSI Blast and ability to produce Smith-Waterman alignments
  • Added support for XML output.
May, 17 2000
  • PSI and PHI Blast were added to this distribution
  • Added support for client/server interface for gi/accession lookups using Entrez
  • Added possibility to print Taxonomy reports
  • Added option to print alternative alignment with specific color schema
March, 20 2000
  • Initial revision

Installation of the Standalone WWW server

After downloading the file wwwblast.Your_platform.tar.gz to your computer, place it into document directory of HTTPD server and uncompress it by 

    gzip -d wwwblast.Your_platform.tar.gz
    tar -xvpf wwwblast.Your_platform.tar

Please note that parameter "p" in tar options is significant - it will preserve file access options stored in the distribution. Temporary directory for BLAST overview images (TmpGifs) should have 777 permission, and logfiles (wwwblast.log and psiblast.log) should have 666.

After you have uncompressed the distribution file, "blast" directory will be created. You can access sample BLAST HTML input forms using URLs:

This distribution comes with 2 BLAST databases: "test_aa_db" - sample protein database and "test_na_db" - sample nucleotide database. These databases are configured to be searchable immediately with compatible BLAST programs.

Description of files in the distribution

Files with suffix "*_cs.*" are analogous to the files without such suffix, with added capability to make client/server Entrez lookups for sequence gis and accessions.

blast.cgi, blast_cs.cgi

- BLAST search start-up C-shell files

.nlmstmanrc

- Configuration file for the graphical overview image (do not edit!)

blast.html, blast_cs.html

- sample BLAST search input HTML forms

megablast.html, megablast_cs.html

- sample MEGABLAST search input HTML forms

rpsblast.html, rpsblast_cs.html

- sample RPS BLAST search input HTML forms

blast.rc

- Default configuration file for the WWW BLAST server

psiblast.rc

- Default configuration file for the PSI/PHI WWW BLAST server

psiblast.cgi, psiblast_cs.cgi

- PSI/PHI BLAST search start-up C-shell files

psiblast.html, psiblast_cs.html

- sample PSI/PHI BLAST search input HTML forms

psiblast.REAL, psiblast_cs.REAL

- Main PSI/PHI BLAST server executables

wblast2.html, wblast2_cs.html

- sample BLAST 2 sequences search input HTML forms

wblast2.REAL, wblast2_cs.REAL

- Main BLAST 2 sequences server executables

bl2bag.cgi

- CGI used to create 2 sequences alignment image on the fly

blast_form.map

- Auxiliary map file for the front BLAST image

nph-viewgif.cgi

- CGI program used to view and delete overview images

readme.html

- this documentation

wwwblast.log

- default logfile

psiblast.log

- default PSI/PHI Blast logfile

ncbi_blast.rc

- sample file for full NCBI set of databases

Configuration of BLAST databases

To set up databases for the standalone WWW BLAST server, it is necessary to follow these steps:

  1. Put a file with concatenated FASTA entries in the "./db" directory
  2. Run "formatdb" program, available from the NCBI ftp site to format the database.
  3. Add name of the database into server configuration file
  4. Add name of the database into (PSI/PHI) WWW BLAST search form

PSI/PHI Blast notes

There is one significant feature of the PSI/PHI Blast server: FASTA files from which the BLAST databases are formatted should have GI numbers in all sequence ids. This is almost always true for FASTA files from the NCBI FTP site. Local databases may not be used with this version of PSI/PHI Blast unless they have ">gi|12345..." prefix in the definition line.

Databases for the PSI/PHI Blast should always be created with formatdb using the "-o T" flag. Test database "test_aa_db" was created using this flag and database "test_na_db" was created without this flag.

If this distribution is installed not in the "/blast" directory under HTTPD documents root directory, than path to the distribution should be set by an environment variable WWW_ROOT_PATH in the file psiblast.cgi or psiblast_cs.cgi

Client/server version for Entrez lookup and taxonomy reports

Regular Blast, PSI/PHI Blast, MegaBlast and Blast 2 Sequences have client/server versions for Entrez gi/accession lookups and printing Taxonomy reports. Configuration of client/server interface from the user to NCBI should be done the same way as for any other client/server program to NCBI. If program "blastcl3" works without problems, this server should work OK as well. If user has firewall - default configuration will definitely fail to work properly and this case will require special configuration. If user has problems with such configuration, one should write to info@ncbi.nlm.nih.gov for further assistance.

XML output

Possibility to produce XML output was added to this server. XML definition of BLAST output is tied to the simple ASN.1 specification designed for this case. These definitions may be found in the directory ./Src/XML. Any recommendations on improvements to this (possible) standard may be sent to blast-help@ncbi.nlm.nih.gov XML may be printed by setting "Alignment view" in blast.html or (blast_cs.html) page to "BLAST XML". Resulting page will look empty - but if you open the page source (in Netscape - View -> Page source) - you will see the complete XML document.

Blast 2 sequences

Blast 2 sequences program was initially written by Tatiana Tatusova and Tom Madden and was presented in the article: Tatiana A. Tatusova, Thomas L. Madden (1999), "Blast 2 sequences - a new tool for comparing protein and nucleotide sequences", FEMS Microbiol Lett. 174:247-250. The standalone WWW version of the program mirrors the NCBI Blast 2 Sequences web page.

RPS Blast

RPS Blast or "Reversed Position Specific BLAST" is a very fast alternative to the program IMPALA. It has the same general objective - to compare a sequence to a collection of conserved domains (aka motifs, profiles, HMMs). RPS Blast has a completely different implementation that has increased the speed of profiles search 10 to 100 times depending on search conditions in comparison to IMPALA. RPS Blast has an option to perform a translated search of DNA sequences against these conserved domains. Currently RPS Blast is one of the tools chosen to annotate human genome at NCBI and is the basis for the CDD Blast search page.

Databases for RPS blast are hardware dependent - for speed reasons. Hence they are different for big/little endian platforms.

To build RPS database it is necessary to follow the procedure explained in the file "README.rps", that comes with this distribution. There is a small RPS database available for testing. This database is a part of a real NCBI database used in CDD search page. Full NCBI database is available in platform-independent form from FTP site.

Out of Frame BLASTX

The OOF version of the blastx program finds alignments between a nucleotide sequence translated into 3 frames and protein sequences, in which query sequence frames can be shifted within one alignment. The algorithm and low-level alignment functions were written by Zheng Zhang, currently at Paracel Inc., and incorporated into regular BLAST API by Sergei Shavirin.

XML and tabular output are not yet implemented for the OOF alignments.

Description of tags for the main BLAST input page

This standalone server has a tag convention analogous to the regular NCBI BLAST server. Sample BLAST search forms may be changed to accommodate particular needs of the user in the custom search. Here is the list of these tags and their meaning. If some tag is missing from the search input page, it will take a default value. Exceptions are tags PROGRAM, DATALIB and SEQUENCE (or SEQFILE), that should always be set.

Server configuration file and logfile

Default configuration file is "blast.rc" and logfile "wwwblast.log". Setting tag BLAST_TYPE to specific value may change these names. This is useful if few different search input pages use the same CGI search engine, but significantly different by content and priorities. A sample configuration file comes with this distribution:

This file will set how many CPUs will be used in the BLAST search and what databases may be used with what programs. Logfile currently stores only limited information but also may be updated by programmers to store more values in it. Please note, that usually HTTPD servers run by accounts that do not have write access to disk, so the logfile permission should be set to 666.

How to debug WWW Blast programs

There is a way to debug these programs.
  1. Add line "setenv DEBUG_COMMAND_LINE TRUE" into the *.cgi file (uncomment it)
  2. Run search that results in the problem - this should create a file "/tmp/__web.in" in the "/tmp" directory.
  3. Set all necessary environment variables on the command line (BLASTDB at least) and run from command-line: "blast.REAL < /tmp/__web.in"
This should do your problematic search without WWW. If this resulted in coredump - you may look into the core file with:

dbx blast.REAL core

and then use command "where" to print stack.

BLAST Help Desk

Last modified: Fri Jan 11, 2002