/MEng/System/CQC/Runtime/HTTPClient

Class Information:

ClassPath:	MEng.System.CQC.HTTPClient
Parent ClassPath:	MEng.Object
Copyable:	No
Final:	Yes

MEng.System.CQC.Runtime.HTTPClient provides CML classes with the ability to make HTTP requests, i.e. to access web based content. The primary purpose of this feature is likely to be 'screen scraping', i.e. making HTTP requests and pulling out information from the returned text which is then made use of. In a driver this data would be stored in drive fields, for instance.

This class also provides a number of helper methods that make it easier to interpret and manipulate the returned contents.

Literals:

Card4 kHTTPCode_OK;
Card4 kHTTPCode_MovedPerm;
Card4 kHTTPCode_MovedTemp;
Card4 kHTTPCode_BadRequest;
Card4 kHTTPCode_NotFound;
Card4 kHTTPCode_ServerErr;

These literals represent some of the most common HTTP status codes. There are quite a few so for others, you will have to define those for yourself in your own classes.

Nested Types:

Enum=HTTPClientErrors
    GetError    : "";
    EscapeError : "";
    ExpandError : "";
    ParseError  : "";
    PostError   : "";
    SetProxy    : "";
EndEnum;

This enumeration represents the exceptions that this class defines. Some don't have any associated text since the text will be gotten from the underlying C++ error that occurred.

VectorOf[KVPair]	LinesList;

This is a vector of KVPair objects. This is used in methods that pass in or out sets of key/value pairs, such as the HTTP header values or GET/POST parameters.

 

Constructors:

Constructor();

There is just a default constructor available.

 

Final, Non-Const Methods:

SendGET
(
    [In]      URL           ToGet
    , [In]    MEng.Card4    WaitFor
    , [In]    MEng.String   Agent
    , [In]    MEng.String   Accept
    , [Out]   MEng.String   RepText
    , [Out]   LinesList     OutHdrLines
    , [InOut] MEng.String   ContType
    , [InOut] MemBuf        Content
    , [InOut] MEng.Card4    ContLen
    , [In]    MEng.Boolean  InBody
    , [In]    LinesList     InHdrLines
)   Returns MEng.Card4;

This method will send an HTTP GET message, and wait for the response to come back. It has quite a few parameters because it is doing a lot for you. The URL is the URL you want to get. It can include query parameters as required. WaitFor is the number of milliseconds you are willing to wait for the response to come back. Agent is just a string that goes into the header that indicates who is asking, so you'd set it to something like "MyMacro/1.0" or whatever you want, making sure it follows the rules for HTTP header line values. Accept is set to the format that you expect the reply to be in. Generally this will be exact, such as "text/html", but the MIME system allows you to provide a less specific type, like "text" and get back any sort of text content.

You can optionally send in body content to be sent out in the GET, as well as get content back in the reply. In many cases a GET has no body, just header lines. But you sometimes want to send a body with it. The InBody parameter indicates whether the ContType, Content, and ContLen fields have valid incoming values to send out in the GET. Else, they are only used to return the reply body to you, if any. The ContType must of course accurately reflect the format and text encoding of the data in Content.

If you want to send out additional header lines beyond the standard ones that are always sent out for you, you can pass them in the InHdrLines parameter.

Those values are then used to build the request and send it. The subsequent parameters are filled in to reflect the successful retrieval of the indicated resource, or the error information that caused it to fail.

RepText is the reply text from the HTTP status line. It'll be something like "OK" or "Not Found" and so forth. It's just for display/logging really, and the Card4 return value is really what you should go by to determine operational status. OutHdrLines is filled in with all of the HTTP header key/value pairs that came back in the response. You may need to examine them to know how to deal with the returned information, or may just want to log some of the information. ContType is the value of the Content-Type: header line. It is also in HdrLines, but is so commonly used that it is pulled out for you as a convenience. ContLen is filled with the number of bytes that gotten written to Content.

Content is a memory buffer into which the raw returned resource data is placed. How to interpret this information depends on the ContType parameter. ContType will be one of the standard HTTP MIME content indicators, such as "text/html" or "image/jpeg" and so forth. It is up to you to determine the type and deal with it correctly.

If the operation fails at the server level, i.e. the server returns an error, then you will just get the returned data back as the output of this method and the return value will indicate the error code that the server returned. If the operation fails at the local level due to bad parameters, inability to contact the server, etc..., then the GetError exception will be thrown.

SendPOST
(
    [In]    URL           ToPost
    , [In]  MEng.Card4    WaitFor
    , [In]  MEng.String   Agent
    , [In]  MEng.String   Accept
    , [In]  ListList      PostVals
    , [Out] MEng.String   RepText
    , [Out] LinesList     OutHdrLines
    , [Out] MEng.String   ContType
    , [Out] MemBuf        Content
    , [Out] MEng.Card4    ContLen
)   Returns MEng.Card4;

This method will send an HTTP POST message, and wait for the response to come back. It is similar in general to the SendGET method above. But this one is designed specifically for the common use of a POST in an HTTP form, where you need to post a set of URL encoded key=value pairs. So you provide those pairs in the PostVals parameter. They are encoded appropriately for you and the encoding it set for you and so on, and the results are sent out. Any reply is returned in the output parameters.

You don't have to pre-escape the key or value strings. They will be escaped appropriately for you and encoded correctly for this standard type of form POST.

If the operation fails at the server level, i.e. the server returns an error, then you will just get the returned data back as the output of this method and the return value will indicate the error code that the server returned. If the operation fails at the local level due to bad parameters, inability to contact the server, etc..., then the PostError exception will be thrown.

SendPOST2
(
    [In]    URL             ToPost
    , [In]  MEng.Card4      WaitFor
    , [In]  MEng.String     Agent
    , [In]  MEng.String     Accept
    , [Out] MEng.String     RepText
    , [Out] LinesList       OutHdrLines
    , [InOut] MEng.String   ContType
    , [InOut] MemBuf        Content
    , [InOut] MEng.Card4    ContLen
    , [In]  MEng.Boolean    OutBody
    , [In]  LinesList       InHdrLines
)   Returns MEng.Card4;

This method will send an HTTP POST message, and wait for the response to come back. It is very much the same as the SendGET method above, except that it sends a POST instead of a GET. So see that method for the details. You can send just header lines by setting the OutBody to False, or you can set it to True and providing incoming values for the ContType, Content, and ContLen parameters, and these will be used as the outgoing body of the POST.

If the operation fails at the server level, i.e. the server returns an error, then you will just get the returned data back as the output of this method and the return value will indicate the error code that the server returned. If the operation fails at the local level due to bad parameters, inability to contact the server, etc..., then the PostError exception will be thrown.

SetProxy([In] MEng.String ProxyAddr, [In] MEng.Card4 ProxyPort)

This method will indicate that you want to send requests through a proxy, not to the host indicated in the URL. So instead of sending the trailing part of the URL to the host in the URL, the full URL is sent to the proxy, which then sends it on to the target and returns the reply. The address can be a DNS name or dotted syntax numerical address.

Final, Const Methods:

CreateBasicAuthStr([In] String Name, [In] String Password, [Out] String ToFill)

Given the passed user name and password, this method will fill the output string, ToFill, with a string that is valid for the HTTP Basic Authentication system. If you get back a 401 error from a GET or PUT, this means that the server is challenging you to log in. If it is asking for basic authentication, you can use this generated string as a header line, with the key "Authentication", and re-send the GET or PUT. If the information is correct, then the operation will be accepted.

If you know that you will be challenged, you can preemptively send this authentication information, and either way you can just send it with each new PUT or GET, to avoid having to do it twice every time. The server won't challenge you if you provide the correct authentication on each PUT or GET. Note that some servers will support more than one 'domain', each of which requires different authentication. If so, you must parse the WWW-Authenticate header line that comes back with the 401 challenge and see what the domain is.

EscapeBodyText([In] String Src, [Out] TextOutStream Target);

The text in the body of an HTML element cannot have certain characters in 'raw' form, because they are special characters used in HTML markup. So these must be 'escaped' using character references. This method will take a source string of text, and an output stream (generally into which the overall HTML page is being formatted), and will format it to the target, doing any required escaping. For the opposite direction, use ExpandBodyText.

ExpandBodyText([In] String Src, [Out] TextOutStream Target);

The text in the body of an HTML element cannot have certain characters in 'raw' form, because they are special characters used in HTML markup. So these must be 'escaped' using character references. This method will take a source string of HTML text that has been parsed out of an HTML document, and format it out to the target string, expanding any character references.

ParseQueryParms([In] String Src, [Out] ItemList ToFill);

When a URL has query parameters, the part after the ?, they must be encoded in a particular way, which makes them hard to deal with for internal purposes. This method will parse a list of query parameters and break them out into a vector of strings, with all escaped characters expanded.

ParseTextEncoding([In] String ContType, [Out] String Encoding) Returns Boolean;

The return from a GET is sometimes a known quantity, i.e. you know what URL you are asking for and what format the data will be returned in. But sometimes you do not. The SendGet() method above will return a ContType value. This is the MIME content type indicated in the returned page. This string will be something like text/html or image/jpg and so forth. If it's text, then it might have an encoding, such as text/html;iso-8859-2, so it's indicating what encoding it returned the text in.

If there is no encoding, then ISO-8859-1 (i.e. Latin1) is the implied encoding and you should assume that. Else, you should use the indicated encoding and transcode the returned buffer based on that. This method will return true if the returned content is text, and return the encoding you should use. If there was no encoding on the ContType it will just return the default Latin1.