| [Top] | [Contents] | [Index] | [ ? ] |
This is the reference manual for the dmbcs-micro-server C++ library (version 0.1, 6 February 2019).
Copyright © 2018 Dale Mellor
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. The text of the license can be found in the section entitled “Copying”.
| 1. Introduction | ||
| 2. Using the library | ||
| 3. Reference | ||
| 4. Examples of Use | ||
| 5. Hints on how the library might be used in a real application |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
This document introduces, defines and exemplifies use of the dmbcs-micro-server C++ library, providing services to applications for processing and responding to requests from a web browser, including the possibility of making the application a stand-alone web server in its own right (a ‘micro-server’).
| 1.1 HTTP Web Services | Introduction to CGI and web servers | |
| 1.2 Our Choice of Name | Use of ‘DMBCS’ | |
| 1.3 Your Choice of Library | Some gentle advice | |
| 1.4 Outline of Library | An overview of the library’s implementation |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
A CGI program is a program, written in any language, which conforms to the Common Gateway Interface (CGI). It is a program executed by a web server (on the server machine), and is passed information about the request from the user through the UNIX environment. It is left to the program to interpret the user’s requirements and to produce on its standard output channel a HTML document which the web server will relay back to the user’s browser for display on his console.
CGI programs are thus very flexible and amenable to lots of jobs. Some of them are very small, amounting to only a couple of lines of an interpreted language to slurp a file from the local storage and send it straight back to the browser (which is essentially what a web server would normally do but this allows the programmer to do so in a more customized way). On the other hand, some CGI programs are extremely large consisting of hundreds of thousands of lines of code which may interact with a database and other third-party systems (such as payment gateways) and do some real work on behalf of the user, before sending the results of the transactions along for the user’s perusal.
This project came from a CGI program of the latter description. Because of its size, complexity and the anticipated load (number of users), it was decided very early on that it would be written in a proper programming language which could be compiled optimized to run on the bare metal of the available computer system, whatever that is. Thus it would have to be C or C++. While the original implementation had been in C for political reasons, it has since been ported to the more convenient C++. As a side-effect of this development, the parts of the system which deal specifically with the business of being a CGI program have been abstracted and encapsulated, and then they were syphoned off into their own separate library; this was further extended to provide the functions of a self-contained web server. This is now being offered for general use.
The result is a library which can be used to inject a self-contained web server into an arbitrary C++ code base, opening up the interface to such a legacy program to the modern world, without needing to also build the infrastructure to serve such a program to the public (although in practice infrastructure needs to be built for other reasons—like security and virtualization—but the self-contained scheme allows for greater flexibility and better functional isolation).
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The micro-server library was written from scratch since 2004 as part of a large internet project. The requirement for a C++ framework for undertaking this task is very common, and thus it is no surprise that other efforts already exist, notably the GNU cgicc project.
Some consideration was given to the name of this project. At first it was going to be called cgicc2, but that carries the implication that it somehow carries on from the GNU cgicc project, and that it is both a development and enhancement of it. This is definitely not the case, but a parallel development, completely different but solving the same problems. Unfortunately appending any number or even letter carries some implication of sequencing, and if the two projects continue (as they should) to be developed then some imbalance would occur in the rivalry. Such is definitely not the intention, but for the two projects to be seen as separate. Choice of one or other framework for any particular project should be based on what the framework offers, not what letters happen to come in its name.
Thus it is decided to mark this particular library with the string ’DMBCS’: all code appears inside the namespace ’DMBCS::Micro_Server’ and the include file is called ’dmbcs-micro-server.h’; the project is formally known as the ’dmbcs-micro-server’ C++ library.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
We offer only this advice: look at the API provided by all the available libraries, and consider the nature of the working framework (paradigm) that that implies; choose the library that seems most appropriate for your project: the one that you will feel most comfortable working with.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The library consists of a high-level wrapper and two workhorse classes.
Http_Server interfaces with multiple clients and delegates
processing of their requests to registered callback functions, which
make direct use of the following workhorses. Query_String looks
after the part of the CGI interface which passes from the browser to the
CGI program and provides methods specifically for interpreting the
arguments from the browser’s query string, and Hyper_Tags looks
after the part of the CGI interface which passes back from the CGI
program to the browser, and displays the results of the user’s request;
this deals with updating template HTML files with some dynamic content
before passing the result to the web server for eventual transmission to
the browser.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
To re-iterate: all code in this library comes in the namespace ’DMBCS::Micro_Server’ and the header which introduces all of the functionality is ’dmbcs-micro-server.h’.
A project would generally consist of a template HTML file with markers where dynamically-generated content needs to be placed, a .css file which should be injected into the file or provided by the server as an inclusion target in the HTML, a .js file containing code that should run client-side to implement part of the functionality of the web site, and a (micro-) web server which runs continually, listens for client connections, and provides generated HTML files and JSON data strings on demand of the client.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
| 3.1 The utility functions | Useful functions needed by web applications | |
| 3.2 The Query_String class | For processing the user’s arguments | |
| 3.3 The Hyper_Tags class | For substituting dynamic content into HTML | |
| 3.4 The Http_Server class | To implement a micro-server in your own app |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Introductory text here.
This function will attempt to open the file named file_name (which may be a complete path to the file), and read the whole lot into a string. If any errors occur in the process, a Slurp_Error object will be thrown (these exception objects do not convey any information back to the application).
This function will attempt to open file file_name (which may be a full path), and will slurp the entire contents (which are presumably a hyper-tag’d template file) into a string. If any errors occur, an empty string will be produced, and a message printed on the standard error channel to indicate the name of the file which has failed.
In the majority of applications of this function, in the majority of calls, the error condition thrown from the function above will not happen; it would be regarded as an infrastructure build problem rather than a run-time error if a known file were not accessible. Thus it is wasteful to worry about and deal with the error at global level–outside the library–, rather let the program carry on and provide a message to the user’s console about the missing file (the fact that the problem occurred will be obvious!) Once it is known which file has failed, the remedy will be equally obvious.
Any occurrence of “ in input will be replaced by ‘"’
All occurrences of HTML-special characters (<>&") will be replaced with entity representations.
The main purpose of this method is to alleviate the possibility of
injection attacks into a web-site; if any part of a page is generated
from user-supplied input there is the danger that the user might be
savvy enough to get JavaScript to execute there! But the function is
quite generally useful when it is known that a string needs to appear
verbatim in a web page, regardless of its contents, and thus ought to be
used whenever a string is added to a set of Hyper_Tags.
Replace all sequences of white space at the start of all lines with a single space character.
HTML recognizes all sequences of white space outside of quotes as a single space separator. Thus sequences of white space at the beginning of lines (which are used by web programmers to give a nice aesthetic indentation) are a waste of bandwidth. This function will remove all such occurrences from input.
Replace all occurrences of quotation marks and escapes with escaped versions, where the escape character is the back-slash.
This function will put an escape (’\’) in front of ”’, ””, or ’\’ which occur in input, and is useful (essential when dealing with user-supplied input) to ensure that strings do not mess up SQL statements.
This function will replace ’+’ with ’ ’, and any %-escaped character codes in the url with the ASCII equivalent.
This will produce a plain string, decoded from strings passed from forms
or EcmaScript on web pages passed through parameter lists in URI’s, as
provided within the context of this library by the Query_String
object.
Return an ASCII string containing the hexadecimal representation of the octets in the given buffer. The buffer_size is the number of octets to decode.
Return the value of the environment variable called env, or an empty string if the variable does not exist in the environment.
This is a convenience wrapper around the system getenv call,
adapting it to the use of strings.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
A Query_String object is a std::map<std::string, std::string> object in which the keys are the variables passed in the URL query string part and the values are the values from that query string. Methods are defined which interrogate the class in details more specific to the CGI problem, and allow for values to be extracted in either string or a numeric form, as expected by the CGI program.
| 3.2.1 MACs | ||
| 3.2.2 Constructors | ||
| 3.2.3 Methods | ||
| 3.2.4 Functions |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The null constructor, only useful if you are intending to build up a new query string for eventual inclusion into some part of a HTML page.
The real workhorse constructor. The argument env should be the environment variable passed to the CGI’s main routine as the third argument. This will create an object which represents the information passed to the CGI program via the query string, whether it were sent via the GET or the PUT protocols. The object returned can subsequently be used to interrogate the user’s request string.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
This method will add an entry into the Query_String with key name and value a string representation of value. The type T of value may be anything that can be passed into a std::ostream object via the << operator.
This method will add an entry into the current Query_String which is a copy of the entry in the input Query_String, unless such entry does not exist in which case a key name with value a std::string representation of fallback will be added to the current Query_String. The type T of fallback may be anything that can be passed to the << operator of a std::ostream object.
Returns TRUE if the query string had been sent to the CGI program via the POST protocol, FALSE otherwise (which case would presumably have been caused by use of the GET protocol).
This gets the value associated with the key name from the query string, unless no such key was present in which case fallback is returned. Note that the type of value returned is the same as that of fallback, thus fallback serves the dual role of defining the fallback value and determining the return type of the function.
In the case that a query string value corresponds to a multiple choice selection on the HTML form, it will be a comma-separated string of selected values. This function will return a vector comprised of these values, interpreted as integers. If there is no key name in the query string, then the returned vector will have no elements (zero length).
This method returns TRUE if the name was a variable present in the query string, FALSE otherwise.
This method will return TRUE only if there was a MAC present in the query string, and the MAC was valid. Otherwise FALSE will be returned.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
This static class member will add a cryptographic MAC onto any string, but this is only meaningful for composed query strings.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Whereas a HTML document contains markup tags expressed as labels between angle brackets (possibly in pairs), so a micro-server template file contains dynamic content holders expressed as labels between square brackets. The Hyper_Tags class contains methods for replacing single tags with some dynamically-generated content, and for selectively keeping or removing sections of the template bracketed between pairs of hyper tags.
A Hyper_Tags object should be considered a container of tag definitions. The CGI program will typically build up a list of such definitions in the container, until all the work is done when the template file is read in from local storage and the Hyper_Tags object is instructed to make all the changes in one go before sending the result to the server and hence to the user’s browser.
| 3.3.1 Constructors_ | ||
| 3.3.2 Methods_ | ||
| 3.3.3 Functions_ |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The class only has the one, default, constructor, which returns an empty object waiting to be filled with tag definitions.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Normally, a later tag definition would overrule any earlier ones when substitutions are finally made into a template file. However, if it is desired that the last tag added should be the final tag definition on substitution, then it can be ‘hardened’ by calling this method.
This method returns TRUE if a tag with the given name has been registered with the Hyper_Tags object. In all other cases it returns FALSE.
This method copies all the tag definitions contained in registry into the current object, and returns a reference to the current object. If the optional harden argument is passed a non-zero value, then all the incumbent tag definitions will be hardened, as per the discussion of the harden_last () method above.
One of the main methods of the class. Declares that, in the final substitutions, any parts of the template file which look like “[name/]” should be replaced with substitution_text.
The return is a reference to the current object.
Declares that tags called name in the template file be replaced with a string representing the integer value. The return is a reference to the current object.
Declares that tags called name be replaced with a string representing the floating-point value, with at least precision significant figures. The return is a reference to the current object.
Declares that tags called name be replaced with a value from q_s with the same name if this exists, or with fallback otherwise. The type T of fallback should be one of std::string, int, or double. The return is a reference to the current object.
Here, I is an iterator type over an STL container of int’s. This method declares that tags named name be substituted with a comma-separated list of integer values.
Specifies that tags named name be replaced with a string representing the date indicated by the UNIX time object. The conversion takes place according to format, which is described in the strftime () libc manual.
Specifies that tags named name in the template HTML file be replaced with the entire contents of the file called file_name in the file system.
The trailing arguments are a list of const char *const, terminated with a NULL pointer. This method specifies that contents of the HTML template file which look like “[pass-through/]” be replaced with a set of lines which look like “<input type=“hidden” name=“name” value=“value”/>” where the names are taken from the list of arguments to the function, and the __values__ are taken from the query_string variables with the same names. Thus, if the tag in the template file is contained within “<form>” markup elements, the current values in the query string will be propagated into any query string that the new form produces.
This method declares the action to be taken when a pair of tags like “[name]” and “[/name]” are seen in a template file. If keep is KEEP_SECTION, then the tags are simply removed from the file leaving the contents between the tags intact. If keep is LOSE_SECTION then the tags and all the text between them is removed from the file.
One of the main methods of the class. This will go through html_template and replace any hyper tags which have been registered with the substitution text that has been declared as the replacement. If a tag is seen in the html_template which has not been registered with the current object, then this tag will remain intact in the template.
Additionally, any pairs of tags seen in html_template which have been registered as a section will be removed, possibly removing also all the text between the tags, according to the registered behaviour.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
It may happen that, after processing, a HTML template string still contains some hyper tags which have not been substituted. This static function will injudiciously remove any such tags, thus eliminating any ugliness from the page sent back to the user.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
| 3.4.1 Constructor__ | ||
| 3.4.2 Functions__ |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
std::map<std::string, std::function<void (Query_String, int)> methods)
Create a HTTP server to listen on the socket. Processing for
requests will be delegated to the methods, referenced by the
HTTP-call path name via the map. When called, the methods
will be passed a Query_String object and a socket on which a
response must be sent; methods are provided in the Http_Server namespace
to facilitate with this.
Please see the example application to see how this is used in practice.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
Cause the server to act on a single instruction from a web client, waiting up to wait_us micro-seconds for such a request to appear.
The return is usually TRUE, but may be FALSE if the
listening socket goes away (this is a mechanism that could be used by a
request processing function to terminate the server frow within).
In practice this method is called repeatedly, ad infinitum.
const std::string &html)
This method sends the html to the socket, dressed up as a valid response to a HTTP request for data.
const std::string &text)
This method sends the text to the socket, dressed up as a valid response to a HTTP request for data.
const std::string &pdf)
This method sends the pdf to the socket, dressed up as a valid response to a HTTP request for data. The ‘string’ is actually understood as a buffer of arbitrary octects here.
const std::string &json)
This method sends the json to the socket, dressed up as a valid response to a HTTP request for data.
This method sends an empty payload to the socket, dressed up as a valid response to a HTTP request for data (it would generally be taken by an Ecmascript application to be a ‘bad’ response to a request for data).
const std::string &png)
This method sends the png to the socket, dressed up as a valid response to a HTTP request for data. The ‘string’ is actually understood as a buffer of arbitrary octects here.
const std::string &svg)
This method sends the svg to the socket, dressed up as a valid response to a HTTP request for data. The ‘string’ is actually understood as a buffer of arbitrary octects here.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
We don’t start with the tradition hello, world program as this would not demonstrate in any way the dynamic nature of CGI programming! The simplest example we can think of allows the user to input two numbers and returns the product of them, and the full code for this is provided in the next section.
| 4.1 Simple example – Online multiplier | A simple application of the library |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The following is provided as the simplest code to demonstrate use of the
dmbcs-micro-server library, not to inform of any coding style or quality
system approach. We assume a standard GNU system with recent
make, bash, gcc, etc.
Start with the HTML file calc.html
<html>
<head><title>Multiplier</title></head>
<body><h1>Multiplier</h1>
<form action="compute" method="GET" id="calc">
<input type="text" id="arg_1"/> x <input type="text" id="arg_2"/>
= <input type="text" id="result">[result/]</input>
<br>
<input type="submit">CALCULATE</input>
</form>
</body>
</html>
And the C++ source file calc.cc
#include <dmbcs-micro-server.h>
using namespace DMBCS::Micro_Server;
/* This function both serves up the basic HTML page, and
* performs the multiplication and injects the result into the
* HTML. */
void home_page (Query_String const &query, int const socket)
{
auto tags = Hyper_Tags {};
add (tags,
"result",
query.get ("arg_1", 0) * query.get ("arg_2", 0));
Http_Server::return_html (substitute (tags,
slurp_file ("calc.html")));
}
int main ()
{
auto server = Http_Server {2022,
{ {"", home_page},
{"compute", home_page} }};
for (;;) tick (server, 1000000);
return 0;
}
then the makefile
CXXFLAGS = `pkg-config --cflags dmbcs-micro-server`
LDFLAGS = `pkg-config --libs dmbcs-micro-server`
Then at the command line type
make calc
./calc
then point a browser at http://localhost:2022/ and use the simple
calculator (don’t try to do anything funny: the code has been kept
deliberately simple and doesn’t do any error checking). Note that an
answer can be obtained directly with a URL like
http://localhost:2022/compute?arg_1=3&arg_2=4.
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
While the basic library as it is serves perfectly well for small CGI programs, the library was derived from a larger project. From this experience, it is seen that many large projects will want to produce specializations of the classes and utility functions provided by the library for the particular project. The following are some specific details about how the library was enhanced in the original project.
| 5.1 Web-site skins | Web sites which provide a user-selected ‘look-and-feel’ | |
| 5.2 Other inheritence | Other ideas |
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
The underlying paradigm promoted by the library is one where the programmer supplies template HTML files which the CGI program populates with extra dynamic content. This opens the possibility of supplying the same dynamic content into different templates, for example where a web site allows the user to choose a favorite ‘skin’, or ‘look-and-feel’, then each template file would exist in different forms to provide all the different skins available. In this case, the library can be extended by ensuring that whenever a template is slurped from local storage for use, it is the template appropriate for the user’s choice of skin that is pulled up.
In the original project, this was achieved by placing all templates for a skin into a single directory to which the CGI program has access (it was actually a sub-directory underneath the cgi-bin directory, and the web server was configured to *not* allow direct access from a browser to these files).
The main modification was to add a new utility function in the project’s namespace
string slurp_file (const std::string &skin,
const std::string &filename)
which is called by the program in lieu of slurp_file, and ensures that the file from the correct skin directory is chosen, calling through to the library function to get the eventual work done. To complete the effect, it is also necessary to derive a new class from Hyper_Tags with the following additional methods
blah, blah, blah
(The Query_String class works completely as is; it may be convenient to introduce this into the projects own namespace with a
typedef Query_String Query_String;
line in a project header file.)
| [ < ] | [ > ] | [ << ] | [ Up ] | [ >> ] | [Top] | [Contents] | [Index] | [ ? ] |
| [Top] | [Contents] | [Index] | [ ? ] |
| [Top] | [Contents] | [Index] | [ ? ] |
| [Top] | [Contents] | [Index] | [ ? ] |
This document was generated by Dale on March 23, 2020 using texi2html 1.82.
The buttons in the navigation panels have the following meaning:
| Button | Name | Go to | From 1.2.3 go to |
|---|---|---|---|
| [ < ] | Back | Previous section in reading order | 1.2.2 |
| [ > ] | Forward | Next section in reading order | 1.2.4 |
| [ << ] | FastBack | Beginning of this chapter or previous chapter | 1 |
| [ Up ] | Up | Up section | 1.2 |
| [ >> ] | FastForward | Next chapter | 2 |
| [Top] | Top | Cover (top) of document | |
| [Contents] | Contents | Table of contents | |
| [Index] | Index | Index | |
| [ ? ] | About | About (help) |
where the Example assumes that the current position is at Subsubsection One-Two-Three of a document of the following structure:
This document was generated by Dale on March 23, 2020 using texi2html 1.82.