Trader Desk Data Service

Market component data server intended to supply the trader-desk application.

1. Introduction

This service is intended to provide the free and open source trader-desk application with timely information about the structure of the worldʼs main financial markets: the components (companies) that make them up. However, the utility of this service is sufficiently broad that it may well be of use to other softwares, and to amateur traders/investors in general. Hence the interface is here publicly documented in the hope that it will be of use to others.

The current service, to be considered beta-grade rather than full production, is mostly hand-curated (there would be no need to have the service in the first place if things were otherwise), though the intention is to automate it more and more as experience of running the service is gained, so that eventually the information can be properly regarded as timely and accurate.

While the server does not, under any circumstances, pass any caller information on to third parties, it uses the data to regulate the service and guard against overuse and misuse. Market components do not change with any great frequency, and so it is not expected that an application will access this service much more than once in any 24 hour period, and when it does it will use the last_time parameter (Section 2.3) to obtain a recent change delta; this service is being provided freely at cost to the developer, please respect it as such.

1.1 Disclaimer

This service is provided as is and no liability is accepted for any loss or damage caused by the use of this facility.

2. The Serverʼs Programmatical Interface

Note that there is a reference implementation of a client-side library for interacting with the server: it is written in C++ and available from https://rdmp.org/dmbcs/market-data-api.

In general, HTTP/1.0 GET requests are made to

https://rdmp.org:9443/trader-desk/{function}?{argument1}={value1}&{argument2}={value2}...

and a response will come back in the form of a HTTP content-type text/plain payload containing a Scheme s-expression which evaluates to a list of data. NOTE well, however, that, although these have the appearance of an s-expression, for security reasons they are not taken as such by applications but parsed as simply formatted UTF-8 strings from which the information is distilled. Thus, apart from white-space and provision for future extensions, the descriptions of the returned values below are extremely prescriptive, and applications can depend on this being the case.

The authors of this service would be hugely disappointed if you were to decide to make life easy for yourself by blithely processing the strings with Guileʼs eval-string function (while being simultaneously ignorant of the TLS server certificate check); that is an invitation to a nasty man-in-the-middle injecting horrible code into your system and thenceforth wreaking havoc onto everyone else. But you are a responsible citizen of the Internet and wouldnʼt contemplate thrusting such ugliness on the rest of the world, right? [We are keeping an eye on the current bleeding-edge Guile 2.2 as this has a sandbox feature, but one which is still unable to protect against memory exhaustion; the advice is currently not to parse server returns directly as Guile scheme.]

Incidentally, access through the TLS protocol is mandatory; for hints how to deal with this in application code read about cURL and the Market_Data_Server class in the trader-desk code).

2.1 Versioning and Extensions

All responses from the service start with a list of version specifications. The first element in the list is the major number, and other elements may be either sub-variant numbers, or symbols (note not lists or cons-cells; the first closing parenthesis found will be the end of the version list). In all cases these represent extra items which may be passed through the interface, and are shown below as __extensions items.

Note that regardless of sub-variants in the extension specification, all extensions will be backwards-compatible with the basic major version, so that applications written for such will always work.

Currently, the only version list the service will return is simply “(1)” and the __extensions data will be absent; they are there to cater for future potentialities. Applications written today should check for base (major) version 1, and refuse the data if anything otherwise is seen.

2.2 Finding out which Markets are being Tracked

A request to https://rdmp.org:9443/trader-desk/markets (once in a while only!) will return a list of tracked markets in the form (except for minimization of white-space)

'(version_list
  ("market_symbol" "component_extension" "short_name" close_time __extensions_1)
  ...
  __extensions_2)

where

version_list	is currently “(1)”,
market_symbol	is the ticker symbol which Yahoo! Financeʼs unofficial market data API (application programming interface)—this will henceforth be referred to as the ‘Yahoo! API’—, uses to identify markets (e.g. “FCHI”),
component_extension	is the symbol which the Yahoo! API appends to component company symbols, after a period (e.g. “PA”),
short_name	is a short human-readable string which describes the market (e.g. “CAC 40”), and
close_time	is the number of hours after midnight until the market closes each working day, expressed as a decimal number of hours (e.g. “16.5”);
__extensions_1	could be anything not containing a closing parenthesis, and,
__extensions_2	could be anything not beginning with an opening parenthesis,

All the markets which the service is currently tracking will be listed.

Note that if the latter extension slot is used, it will never be a list and hence will not open with a parenthesis; that would be just too confusing and break backwards compatibility with the base version 1. In other words, applications designed to use the present version of the protocol should assume that the market list continues until such time that no more opening parentheses are found.

2.3 Finding the Current Components in a Market

A request to

https://rdmp.org:9443/trader-desk/components?m={market_symbol}&t={last_time}

will produce a list of component changes in the market with the market_symbol which was returned from the markets function, which have become known to the service since last_time, a long integer representing UNIX time: seconds since the start of the year 1970. Note the importance of the underlined terms: these time-stamps do not correspond to movements in the markets directly, but to changes in the records which this service keeps.

The intention is that an application will make note of the time when this function is accessed for a particular market, and then will pass this time back into the function at the next call so that the return will represent the changes that have occurred in the intervening period.

The format of the return is, (but with minimal white-space),

'(version_list
  ("company_symbol" "company_name" action __extensions)
  ...
  __extensions)

with

Thus the return represents changes to the markets, for the sake of some network and server efficiency.

Clearly, all components currently in the market will be returned if last_time is zero. Otherwise, do not be surprised if the component list comes back empty, as changes to markets are relatively rare events.

As an example, you can look at
https://rdmp.org:9443/trader‐desk/components?m=FCHI&t=0 (once in a while only!)

3. Contact and Bug Reports

Problems, requests for help, issues with this documentation or the web site generally, plaudits, praise and general requests for contact with the authors should be sent through the contact page.

You may also request to be sent an e-mail whenever a new release is made available, or more generally whenever there is news about the project; let us know what you desire.

4. Donating

Human resources aside, the cost of running this service is about 0.1 bitcoin per year. If you feel that this is a valuable resource and you would like to contribute to the cost, and help to sustain the vitality/longevity of the project, please consider sending some small amount of bitcoin to the wallet 1PWHez4zT2xt6PoyuAwKPJsgRznAKwTtF9 .

Thank you.