This section contains information about CGI programming with C and C++ including annotated code from the example in the text.
Common Gateway Interface (CGI) programming is the classic mechanism for dynamically generating response pages for requests submitted by browsers. Data entered in HTML forms, as shown in a browser, are routed to a particular CGI program running on the web-server host machine. The CGI program processes the submitted data, generally updating server-side files or database tables, and then generates the response page that is to be routed back to the client's browser.
The CGI specification defined how data were to be passed between a web-server program, such as Apache, and another data processing program which, run as as a separate process, generates a dynamic response page. Data are transferred from the web-server to the processing program in two ways:
(A "pipe" is an in-memory data buffer; one process can write to a pipe, another process can read from the pipe.)
The text of the dynamically generated page is written to the processing program's standard output stream. This data stream passes through another "pipe" back to the web-server. The returned data include some of the HTTP header information for the response; in particular, the returned data need start with a specification of the content type (text/plain, text/html, image/gif, ...). The web-server adds other headers and then returns the response to the browser client.
Environment variables have long been a standard part of the Unix and
Windows systems. They are name-value pairs (both name
and value being simple strings). Their role is to set
the many optional parameters that control the behavior of
other programs launched from a command shell. On Unix you
can view your environment through the use of a command such
as env; on a Windows system, the command set
given in a Command Prompt window provides similar data.
You will see that your environment includes things like PATH
(the value for this variable on both systems is the set of directories checked
for programs that are to be executed when commands are entered in a shell window
or command prompt window). Other variables will include things like your
computer's name, possibly a prompt string for a command window, default directories,
maybe Java classpath settings, and parameters used by database
systems.
When one program launches ("forks") another, it can set the environment variables that are seen by the subprocess. So, a web-server like Apache can add data to the normal environment, making these additional data available to whatever program runs in the new subprocess.
A process can pick up the values of environment variables using
a system call such as char *getenv(const char *name) (this
function is in C's <stdlib.h>). The function returns the
string value for the named environment variable (or NULL if
the specified argument name does not occur in the environment).
The CGI specification defined a set of extra environment variables
whose values were to be available to a launched CGI program. These
included variables whose values characterized the web-server (e.g
SERVER_SOFTWARE and SERVER_NAME); data
characterizing the client (e.g. HTTP_USER_AGENT, REMOTE_ADDR, and
REMOTE_USER); and data characerizing the request
(e.g. HTTP_ACCEPT, REQUEST_METHOD,
PATH_INFO, and for requests with entity bodies (PUT and POST
requests) CONTENT_LENGTH and CONTENT_TYPE).
The QUERY_STRING environment variable contains
all data submitted as a query string appended to the URL that
identified the processing program. These data will be the
data entered in a form that is submitted using an HTTP GET request.
The use of "pipes" is largely hidden in the low-level system code. They simply provide a mechanism whereby the web-server can write data that the subprocess can read from its standard input (C's stdin, Perl's STDIN, C++'s cin), and similarly allow the subprocess to write its response text page and have this read by the web-server.
Setting up "pipes" requires a bit of messy but standardized system code. All this work is done in the web-server. The code for the program run as a subprocess is quite simple; it is just a normal program that reads from stdin and writes to stdout.
Any data in an "entity" body for a POST or PUT request are written by the web-server to the pipe. The CGI program can read these data on stdin.
Data entered in a HTML form are encoded before they are transferred via HTTP. This is not a security encryption; it is simply a character substitution process that avoids problems with characters whose presence would cause problems to the transfer protocol. (For example, an address entered in a form could include new-line characters; but these new-lines cannot be included in a "query string" because that would disrupt the structure of the HTTP header.)
The encoding scheme is quite simple:
A CGI program must reverse this encoding before it can process the submitted form data.
Each input field in a HTML form should have a name. The browser sends the field's name and the associated input data as a name=value pair. The name and value strings are encoded; ampersand characters are used to separate the pairs.
In the case of the example (Pizza order form, the submitted data will include a single value for the pizza size (form input element being one of the p_size radio buttons), several values for options in the pizza toppings section ("tops" selection input), and text data taken from a text input fields such as "customer" and "address".
If for example, a customer ("D.H. Smith", at "24 Bomba Street.\nDapto.") picks a Family sized pizza with Pepperoni, Olives, and Sun dried tomatoes, the query string ("GET" request) or entity body ("POST" request) will contain data:
p_size=fam&tops=Pepperoni&tops=Olives&tops=Sun+dried+tomatoes&customer=D%2eH%2e+Smith&address=24+Bomba+Street%2cDapto%2e
A CGI program starts with its data in this kind of string. It must separate the name=value pairs, identify the names and values (with appropriate decoding applied to the encoded strings).
CGI programs all have essentially the same work to do and so their basic structure is along the following lines:
USER_AGENT), or
you might want to vary a response according to a language preference.QUERY_STRING environment
variable depending on the REQUEST_METHOD). These
data are used to fill in the fields of the allocated structure; the
input field names identify the structure fields.Much of the code is standardized; in particular, the code for parsing a string or stream with (name, value) pairs is fixed and independent of the particular application.
The application specific parts involve:
The standard code for finding and returning name value pairs can be factored out in many ways. There are several libraries of C/C++ code that can be used (including the W3C's own code). The example code has the standard code implemented in functions of an abstract C++ base class; the class has virtual functions, such as a function for processing a name-value pair, that must be overridden in concrete subclasses.
The example C++ code is based on two classes:
while (more data) { extract next
name value pair as a token, process the token }.A program for processing a specific HTML form page is
implemented by defining a subclass of CGI_Helper. This
subclass must at least override the empty ProcessToken
virtual method in the base class (this is the function that does
something with a Token
containing the next name-value pair in the input).
The subclass must either add member functions that do the work
of data validation and response generation, or incorporate such
code in an overridden version of the virtual EndTokens
member function of the base class.
The programs have been run when compiled with the Sun Solaris CC compiler and the g++ 2.97 compiler. Later versions of the g++ compilers may generate many warnings because the code utilizes the old style #include headers (e.g. #include <stdlib.h>). The programs have not been tested with Microsoft's C++ compiler for Windows.
An annotated versionof the code is available. The class definitions are:
class Token {
// Token, a name, value pair
// name will be field name for element of form on HTML page
// value will be content
// (if have multiselection list, you may get many tokens with
// same name)
public:
Token(char *name, char *value);
~Token();
const char *Name();
const char *Value();
private:
char *fName;
char *fValue;
};
class CGI_Helper {
public:
CGI_Helper() { }
virtual ~CGI_Helper() { }
// Header, Trailer functions simply compose some
// standard HTML code for the generated response page
// (written to stdout, will be returned by Webserver)
virtual void HTML_Header(const char* title);
virtual void HTML_Trailer();
// Handle request
// determines if Get or Post, loads data appropriately
// then using Process(), it loops through all tokens in
// request
virtual void Handle_Request();
protected:
// these two for convenience, you might find it useful
// to have something done by CGI_Helper before/after tokens
// called from Process()
virtual void StartTokens() { }
virtual void EndTokens() { }
// Process --- that loop through tokens
virtual void Process(char *data);
// Override this,
// typically create a CGI_Helper with link to some other
// object, ProcessToken can then pass each token to that object
virtual void ProcessToken(Token *tok) { }
private:
int hexvalue(char ch);
void ReadString(char *str, char*& data, char*endpt);
Token *GetToken(char*& data);
void HandleGet();
void HandlePost();
};
A program using a class based on CGI_Helper will instatiate an
object of the derived class, invoke its HTML_Header
function (the argument is the title for the returned web page),
then invoke the object's Handle_Request method.
The Handle_Request sorts out whether it is
a GET or POST request, finds the form data (QUERY_STRING or
stdin), and then runs the loop extracting name value pairs.
(This processing utilizes the non-virtual private functions
that deal with issues like decoding of escaped characters.)
The processing loop invokes the virtual ProcessToken
method for each isolated token; the code for this
method, asdefined in the subclass,
deals with the data. Finally, subclass specific methods
must be invoked to validate and process the received data.
The Echo server example utilizes this framework to create a CGI program that merely echoes each name-value pair received from the client browser. These data are printed as a simple ordered HTML list (<ul> <li>name 1 value 1</li> <li>name 2 value 2</li> </ul>).
The Start_Tokens member function of the
base class is overridden to print the ordered list header.
The ProcessToken function prints one list item.
The End_Tokens function prints the list footer.
class EchoCGI : public CGI_Helper {
protected:
// Here define some actual processing functions
// for members that in the base CGI_Helper class are 'do nothing'
// functions
virtual void StartTokens() ;
virtual void ProcessToken(Token *tok);
virtual void EndTokens();
// a real processing class would have some data members that
// would hold the data read and an extra public function
// that could be invoked to process the data and generate
// the body of an HTML reply
};
void EchoCGI::StartTokens()
{
// Called before first token is processed
// here want some extra html output
cout << "<ol> " << endl;
}
void EchoCGI::EndTokens()
{
// called after last token processed
cout << "</ol> " << endl;
}
void EchoCGI::ProcessToken(Token *tok)
{
// Usually, you would use token to update a data structure
// Here, simply want to echo data to user
cout << "<li> " << tok-> Name() << "\t: ";
if(tok-> Value() == NULL) cout << "(not specified)";
else cout << tok-> Value();
cout << endl;
}
The Echo server CGI program instantiates an EchoCGI object and uses its member functions to process incoming data:
int main()
{
EchoCGI x;
// the HTML_Header function of class CGI_Helper outputs standard 'http'
// protocol header lines, the argument is a page title
// also outputs some standard html code, from < html> < head> ...
// through to < /head> < body>
x.HTML_Header("Echoing your name value pairs");
// the Handle_Request function keeps calling ProcessToken, once
// for each Token (name, value) that it reads
x.Handle_Request();
// If you had been building a structure to hold the (name, value) data, this
// would be a good spot for some processing code!
// the HTML_Trailer function outputs trailing HTML code like < /body> < /html>
x.HTML_Trailer();
return 0;
}
The program can be built using a simple makefile such as the following:
CC=g++ CCFLAGS= LDFLAGS= P1FILES=CGI.o echo.o echo.cgi: $(P1FILES) $(CC) -o echo.cgi $(P1FILES) $(LDFLAGS) CGI.o: CGI.h echo.o: CGI.h
The built echo.cgi program would then have to be moved to a cgi-bin directory of
a web-server (or another directory where cgi scripts are permitted). It could then
be accessed via any form page where the action attribute pointed to
the echo.cgi program.
Annotated code is provided for this example. The program is a slightly more realistic illutration of CGI processing. The data from the Pizza order form are processed by:
It is still a simplified example; there is really no data validation performed. The program attempts to append log data to a text file. If this file cannot be opened for update, the program will continue to run and generate a response page but make not permanent record.
If you are experimenting with your own Apache, it will run with your own user-id and will have no problems with read/write access to any of your scripts and data files. A real server, running at privileged port 80, will run using some special user-id; typically "www" or "nobody". Its access to files is then limited.
You can make files readable and executable for such a server by simply setting the appropriate permission bits - you have to grant "global" read and execute permissions.
What about a file that has to be updated, such as a log file for a CGI program? One thing that you do NOT do is grant global write permission on a file that you own (after all, setting a global write file permission says that you want everyone else to have the right to trash your data).
If a CGI program is to update one of your files, then on Unix/Linux it must run with your user id as its effective user id. The CGI program is still going to be run by "www" or "nobody" but it runs with your user id as the effective user id and so has your rights of access to your files.
On Unix/Linux, there are a couple of mechanisms whereby a process can
obtain a different effective user id. One works through the file system. Executable
files, like compiled and linked C++ programs, or scripts for an interpreter like
Perl, can be marked as "set user id". When the OS launches such an executable,
it changes the effective user id of the process to that of the file owner. So
all you need to do is set the "set user id" status bit in the directory entry
for your executable (this is an option in the chmod command that
changes file attributes in Unix). Unfortunately, your system's adminstrator
may have configured your Unix system so that these set user id status bits are
not honored.
There is another mechanism whereby a privileged process can itself change its effective user id. This is used in the SUExec wrapper for Apache. This is really too advanced for students to play with and does require privileges.
If you do have file access problems, you will probably need to get your local systems administrator to find a solution.
It is unsual for a CGI program to use files for its persistent data, almost all use relational databases. This immediately overcomes problems with file access. A CGI program will contain the user name and password for a database account and use these when opening a connection to a database. It doesn't matter that the CGI program is being run by "nobody" - it has quoted a valid user name and password for the database.
Those with Java experience (java.sql) will be familiar with the relatively easy mechanisms that exist for database accesses. You open a connection to a database by selecting a driver and specifying a URL, a user name and password. You can then get your "Connection" object to "prepare statement objects" that will be used to run updates and submit requests. The code is all database neutral (provided that you don't try to do anything too sophisticated in your SQL). If you need to switch databases, it is simply a matter of changing a database driver and a URL. Perl's solution is very similar.
The mechanisms used with C and C++ were developed earlier and are somewhat clumsier. One approach is to use database specific application programmer interfaces (this kind of approach is illustrated in some of the PHP examples in the text). However, this requires that you re-write your program if you change your database.
A slightly better alternative is to use "embedded SQL". This technology is covered by international standards and a number of database systems - Oracle, Postgres, DB2, etc - support embedded SQL. You don't write the calls to the functions defined by the database specific API - they are written for you by a pre-compiler.
Your source program contains SQL code in EXEC SQL fragments interspersed
through your C or C++ code. This source code is run through a pre-compiler that replaces
the embedded sql statements with variable declarations and function calls. Each database
vendor supplies their own precompiler. Once the code has been through the precompiler (and
typically has expanded by about one order of magnitude in length) it can be compiled with
the normal language compiler. Ideally, all precompilers will accept the same
embedded SQL statements.
The result is source code like the following fragment that comes from a version of the pizza order program that is modified to work with embedded SQL and an Oracle database:
// A few type declarations
typedef char asciz256[256];
typedef char asciz33[33];
typedef char asciz7[7];
EXEC SQL BEGIN DECLARE SECTION;
EXEC SQL TYPE asciz256 IS STRING(256) REFERENCE;
EXEC SQL TYPE asciz33 IS STRING(33) REFERENCE;
EXEC SQL TYPE asciz7 IS STRING(7) REFERENCE;
EXEC SQL END DECLARE SECTION;
EXEC SQL INCLUDE sqlca;
...
// a function that connects to a database
void connect(const char* name, const char* password) throw(Problem)
{
EXEC SQL BEGIN DECLARE SECTION;
asciz33 dbuser;
asciz33 dbpassword;
EXEC SQL END DECLARE SECTION;
strlcpy(dbuser, name, 33);
strlcpy(dbpassword, password, 33);
EXEC SQL CONNECT :dbuser IDENTIFIED BY :dbpassword;
// code checking for errors
...
}
// A function to insert a record in a table
void recordOrder(
const char* customer,
const char* address,
const char* phone,
const char* email,
int pizzasize,
int delivery,
int toppingsCode,
int extrasCode,
float cost) throw (Problem)
{
EXEC SQL BEGIN DECLARE SECTION;
asciz33 _customer;
asciz256 _address;
asciz33 _phone;
asciz33 _email;
short _size;
short _deliv;
short _extras;
short _toppings;
float _cost;
EXEC SQL END DECLARE SECTION;
strcpy(_customer, customer);
strcpy(_address, address);
strcpy(_phone, phone);
strcpy(_email, email);
_size = pizzasize;
_deliv = delivery;
_toppings = toppingsCode;
_extras = extrasCode;
_cost = cost;
EXEC SQL INSERT INTO PIZZAORDER VALUES
( :_customer, :_address,
:_phone, :_email,
:_size, :_deliv,
:_toppings, :_extras,
:_cost
);
// code to check for errors
...
EXEC SQL COMMIT;
}
It is a bit clumsy, but it is usable. If you have access to Postgres, DB2, or Oracle, the documentation will include examples illustrating embedded SQL and the use of the precompiler. You might encounter this technology in a database course. It is not something that you really want to work with in a server technology subject.