Sections description

General section

The global section gives access to configuration options that affect the overall operation of the proxy server.

Table 5-1. General options

OptionTypeDescription

Hostname

StringThe hostname of the proxy, this needs to be configured properly for CARP and Web interface requests through HTTP to work.
Connection timeoutINTThe timeout in seconds to wait for a connection to be established before giving up.
Header timeoutINTThe timeout in seconds to wait for a client to make the initial HTTP request.
Keepalive timeoutINTThe timeout in seconds to wait for keepalive requests.
Maximum buffer sizeFILE_SIZEThe maximum size in bytes of files that are buffered and processed by the rewrite, keyword, and external features.
Temporary directoryStringThe directory temporary files are stored in.
Web interface line lengthINTThe maximum length of a string with no spaces until an explicit break is placed in it, this is needed since lines without spaces won't wrap in a table, causing formatting problems.
CONNECT portsPORT_RANGE_LISTThe ports outgoing CONNECT requests are allowed to be made to;
Connection pool sizeINTThe number of keep-alive connections to HTTP and FTP servers to keep in the connection pool; these connections will be shared between threads.
Connection pool timeoutINTThe time in seconds a connection may remain in the connection pool before being removed.
Always compress mimetypeStringA regular expression matching the MIME-types which should always be buffered and compressed even if they wouldn't be buffered otherwise.
Compress outgoingBooleanToggle gzip or deflate encoding of outgoing processed content if the client supports it. If the proxy server is running locally, it is recommended you disable this feature.
Compress incomingBooleanThis option will make Middleman attach an Accept-Encoding header that lets the Web server know we can accept gzip and deflate content encodings regardless of whether or not the browser making the request supports it; if the browser doesn't support it, it will be buffered and decompressed before sending.
Add X-Forwarded-For headerBooleanThis option will add a header letting an upstream proxy or Web server know the IP address where the original request came from.
Add Via headerBooleanThis option will add a header letting an upstream proxy or Web server know what proxy server the request passed through.

FTP section

Table 5-2. FTP options

OptionTypeDescription
Passive modeBooleanUse passive mode for FTP transfers; this is useful if you are behind a firewall that prevents the FTP server from opening a connection to you.
TimeoutINTThe timeout to wait for a response to commands sent to the FTP server.
Anonymous loginStringThe login to use when none is explicitly given in the URL.
Anonymous passwordStringThe password to use when none is explicitly given in the URL.
Sort orderRadio buttonsThe order FTP directory listings are sorted.
Sort fieldCheckboxesThe field which FTP directory listings are sorted

DNSBL section

Table 5-3. DNSBL options

OptionTypeDescription
TemplateStringThe template to send when domain is found to be blocked.
DomainStringThe domain to prefix the domain being checked to; i.e. in.dnsbl.org will cause a lookup for bad.com.in.dnsbl.org to be made when a page from the bad.com domain is requested.
Blocked IP addressesIP_RANGE_LISTA comma separated list of IP addresses that can be returned when doing the DNS lookup which will cause the page to be blocked.

Prefetch section

Table 5-4. Prefetch global options

OptionTypeDescription
ThreadsINTThe number of threads to run in the background prefetching files. Middleman needs to be restarted for this setting to take effect.
Queue sizeINTThe size of the prefetch queue.

Table 5-5. Prefetch entry options

OptionTypeDescription
Tag nameStringThe HTML tag the attribute is in.
Tag attributeStringThe HTML tag attribute holding the URL to be prefetched.
Attribute patternStringA regular expression matching the attribute value this entry applies to.
Maximum file sizeFILE_SIZEThe maximum size of the prefetched file.
Recursion levelINTIf the URL leads to another HTML page, this is the depth links will be followed. Setting to 0 causes links to be followed indefintely.

Network section

The network section is used to configure general network settings. The configuration file must be saved and the proxy server has to be restarted before any changes take effect.

Table 5-6. Network options

OptionTypeDescription
PortINTThe port number to listen on.
IPINTThe IP address of the interface to bind to; leave empty to have the proxy listen on all interfaces.

Profiles section

The profiles section allows groups of profiles to be enabled or disabled based on the URL being requested. This is useful to enable or disable groups of related entries in other sections together.

Table 5-7. Profiles options

OptionTypeDescription
ProtocolStringA regular expression matching the protocol this entry applies to; leave empty to match everything.
HostStringA regular expression matching the host's this entry applies to; leave empty to match everything.
FileStringA regular expression matching the file's this entry applies to; leave empty to match everything
Mime typeStringA regular expression matching the MIME-type's this entry applies to; leave empty to match everything. MIME-type matching is done after receiving the server header, so it may only be used for certain features; header filtering, cache refresh policy, and cache store selection are done before the server header is received.
Port range listPORT_RANGE_LISTA comma seperated list of ports or port ranges this entyr applies to.
Added profilesSTRING_LISTA comma separated list of profiles to add when this entry matches.
Removed profilesSTRING_LISTA comma separated list of profiles to remove when this entry matches.
URL CommandSTRING_LISTA comma seperated list of URL commands which will activate this entry. If left empty, this entry is enabled regardless of which URL commands are used.

Access section

The access feature is used to control who can access the proxy server, and to what extent.

Table 5-8. Access global options

OptionTypeDescription
PolicyRadio buttonsDefault action to take when no matching entry is found.

Table 5-9. Access entry options

OptionTypeDescription
IP AddressStringA regular expression matching the IP addresses this entry applies to, leaving blank will cause the entry to match everything.
PAM authenticationBooleanIf this option is selected, clients will be required to authenticate with the proxy and PAM will be used to authenticate the username and password. This option will work only if the proxy is configured and compiled with PAM support.
User nameStringIf this field is not empty, clients matching this entry will be required to authenticate with the proxy server. There can be more than one entry matching the same IP address, in which case the one matching the username/password send by the browser is used. This option is a regular expression.
PasswordStringThe client's password if the username field is used.
AccessCheckboxes

A list of features that connections matching this entry are allowed to access, the options are:

  • Web interface - Access to all of the web interface (access to /mman/template/<template name> is always allowed regardless of this)

  • Proxy requests - Allowed to make regular proxy requests

  • CONNECT Requests - Allowed to make CONNECT requests

  • Transparent proxying - Allowed to make transparent proxy requests (must be allowed to make HTTP requests as well)

  • HTTP Requests - Allowed to make regular HTTP requests to proxy (for Web interface and redirected requests)

  • Allow bypassing - Allows features to be bypassed by prefixing with URL command

  • URL commands - Allows use of URL commands

BypassCheckboxesA list of features which will by default be bypassed when making requests.

Cache section

The caching section is used to configure global cache options and to add/remove cache storage spaces.

Table 5-10. Cache global options

OptionTypeDescription
Violate RFCBooleanThis option will cause the proxy server to violate some rules in the HTTP RFC to help improve cache performance; specifically, when a website requests that the file not be cached with the “No-Cache” directive in the Cache-Control header, the proxy will cache it anyways but always validate it with an If-Modified-Since conditional request.
Memory cache sizeFILE_SIZEThe maximum size in bytes of the memory cache.
Memory free extraFILE_SIZEThe number of additional bytes to free up when the memory is cleaned.
Minimum file sizeFILE_SIZEThe minimum file size in bytes of any cached file.
Maximum file sizeFILE_SIZEThe maximum file size in bytes of any cached file; if set to 0, no maximum file size is imposed.
Maximum wait sizeFILE_SIZEIt's not possible with the current design for one thread to download a file being cached by another thread, but that thread can wait for the other to finish and then use it; this option will configure the maximum size the cache file can get before it stops waiting and transfers the file directly from the server.
Prefetch windowINTThis option can be used to specify the time period after a file is prefetched in which it will be exempt from any refresh or expiry rules.
ICP portINTThe UDP port to listen for ICP packets on.
ICP timeoutINTThe timeout in milliseconds for response ICP packets.
Store balance methodRadio buttonsThis option controls how the storage directory a file goes into is selected. Fill size will select the storage directory with the least total bytes used, Fill percent will select the storage directory with the lowest percentage of space used.

Table 5-11. Cache store entry options

OptionTypeDescription
PathStringThe directory where cached files are stored.
Maximum disk sizeFILE_SIZEThe amount of space that should be used to store cached files in this directory.
Disk free extraFILE_SIZEWhen the cache is cleaned, this additional amount will be freed as well. This option can be useful to prevent the cache from getting evicted too often, which can hurt performance.

Table 5-12. Cache refresh entry options

OptionTypeDescription
CachableBooleanWhether or not requests matching this entry are cached.
Minimum ageUINTThe minimum age any file must be according to the Last-Modified header before it is cached.
Maximum ageUINTThe maximum age of any cached file before it must be revalidated; this overrides any given expiry time.
Revalidate ageUINTThe maximum age of any cached file which didn't include any headers that indicate when it should expire before it must be revalidated; if set to 0, all cached files whose expiry time is uncertain will be verified. If no "Last-Modified" header is received to calculate the percent of age freshness, the cached file is always revalidated.
Last-Modified time factorUINTThe percentage of time between the date given in the Last-Modified header and the current time a cached file will be considered fresh after downloading.

Templates section

Templates are used throughout Middleman as a replacement for pages which can't be displayed due to filtering, error, or other conditions.

Table 5-13. Templates global options

OptionTypeDescription
PathStringLocation to look for templates in if no absolute path is given.

Table 5-14. Templates entry options

OptionTypeDescription
NameString

The name of the template, this is used in other sections to reference it. It may also be one of the following to replace internal error messages:

  • blocked - Page blocked

  • nodns - DNS lookup failed

  • badrequest - Malformed HTTP header from client

  • badresponse - Malformed HTTP header from server

  • nofile - File not found

  • nocache - Cache file not found when browsing in offline mode

  • noconnect - Connection failed

  • noaccess - Access denied

  • badprotocol - Protocol not implemented

  • badauth - Authorization failed (when forwarding through SOCKS4)

  • maxbandwidth – Bandwidth limit exceeded

  • maxrequests – Request limit exceeded

  • proxy.pac – A script to configure the browser to use the proxy.

There are 3 built-in templates that can be used: tinygif (a 1x1 transparent gif image), checkedgif (a 4x4 gray and transparent checkered pattern), and tinyswf (an empty flash animation). You can override the content sent by a website for certain response codes by making a template with a numerical name the same as the response code. There are several variables that can be used in templates if the parsable option is selected which will be replaced with information about the request currently being handled, they are:

  • $HTTP_METHOD - Method used to request file

  • $HTTP_HOST - Host HTTP request was made to.

  • $HTTP_FILE - File HTTP request was made for.

  • $HTTP_PORT - Port HTTP request was made to.

  • $IP - IP address of client making request.

  • $INTERFACE – IP address of the interface the client connected to.

  • $PORT – PORT the client connected to.

Templates can be accessed directly by loading "http://mman/template/<template name>".

FileStringThe filename of the template
Mime typeStringThe MIME-type of the template. When using an executable, this can be set to STDIN to have the MIME-type extracted from a "Content-type" header sent by the program, this will be explained in greater depth below.
Response codeINTThe response code to use when sending the template, leave blank to use internal default.
TypeRadio buttonsTemplate type, either File or Executable. If executable is choses, the file is executed and whatever it writes on STDOUT is sent as the template. Several environment variables are set for the executable to use, they will be explained further below in the external section.
ParsableBooleanIf this option is selected, all variables in the template will be substituted.

MIME section

The mime feature allows you to filter content based on it's MIME-type.

Table 5-15. Mime global options

OptionTypeDescription
PolicyRadio buttonsThe action to take when no matching entry is found.
Default templateStringThe template to send for blocked MIME-types if the template option is left blank for the matching entry, or if no matching entry is found but the policy is deny.

Table 5-16. Mime entry options

OptionTypeDescription
HostStringA regular expression matching the host's this entry applies to, leave blank to match everything.
FileStringA regular expression matching the file's this entry applies to, leave blank to match everything.
Mime typeStringA regular expression matching the MIME-type's this entry applies to, leave blank to matching everything.
TemplateStringThe template to send when an entry matches, this has no purpose in entries with the action set to allow.

Redirect section

The redirect feature allows you to redirect requests.

Table 5-17. Redirect entry options

OptionTypeDescription
URLStringA regular expression matching the URL's you wish to redirect; the URL will always be in the form "protocol://host/file" or "/file" for HTTP requests.
RedirectStringThe URL to redirect to; it may contain back references to strings captured using parenthesis in the URL pattern. This can be in the form "protocol://host/file", or "/file" if you wish to send a relative URL when redirecting a URL in the Location: header. If this option is left blank, no action will be taken against requests matching the URL. See the rewrite section for additional notes on using regular expressions with back references.
PortINTThe port to redirect to; if left blank the same port the original request was made to is used.
302 RedirectBooleanIf yes, a 302 redirect is issued; otherwise the new host is connected to directly and the new file is requested. A 302 redirect should always be used when possible to ensure relative links and images are correct.
OptionsCheckboxes

Several options are available to control how the URL should be handled, they are:

  • Encode URL - Encode the new URL.

  • Decode URL before - Decode the URL before attempting to match it with the regular expression

  • Decode URL after - Decode the new URL after matching.

Applies toRadio buttonsThis option is to choose whether the redirection applies to requested URL's, the Location header when a remote site sends a 302 redirect, or both.

Forward section

The forward feature allows you to selectively forward requests through another proxy or SOCKS4 firewall based on their URL.

Table 5-18. Forward global options

OptionTypeDescription
Enable CARPBooleanEnable Cache Array Routing Protocol.

Table 5-19. Forward entry options

OptionTypeDescription
ProxyStringThe hostname or IP address of the proxy to forward through; if this is left blank, and the host or file options aren't, no action will be taken for requests matching the host and file.
User nameStringThe username to use if the proxy requires authentication.
PasswordStringThe password to use if the proxy requires authentication.
DomainStringThe NT domain when using the NTLM authentication protocol.
PortINTThe port number of the proxy to forward through.
ICP Peer typeRadio buttonsThe peering releationship of this proxy, see “Internet caching protocol” section below for further details.
ICP PortINTThe UDP port ICP packets are sent to.
TypeRadio buttonsWhat type of proxy to forward through; can be HTTP, SOCKS4, or CONNECT.
Applies toCheckboxesWhat type of requests are forwarded; can be HTTP and/or CONNECT (HTTPS)

Header section

The header feature allows you to control what headers are passed from your browser to websites. In additional to the allow and deny actions in some other sections, there is an insert action which will add a new header onto the ones sent by your browser; for these entires, the Type and Value options are plain text.

Table 5-20. Header global options

OptionTypeDescription
PolicyRadio buttonsThe action to take when no matching entries are found.

Table 5-21. Header entry options

OptionTypeDescription
TypeStringA regular expression matching the header type's this entry applies to; leave blank to match everything (header's are in the form "Type: value").
ValueStringA regular expression matching the header value's this entry applies to; leave blank to match everything.
Applies toCheckboxesThis option is to select whether or not this entry applies to the server header, client header, or both.

Rewrite section

The rewrite feature allows you to use regular expressions to modify the contents of web pages, files, the client header, and server header.

Table 5-22. Rewrite entry options

OptionTypeDescription
Mime typeStringA regular expression matching the MIME-type's this entry applies to. This must be filled with something, otherwise the rewrite rule will be applied to every downloaded file, which is almost certainly not what you want. To have it applied to web pages, fill this field with "text/html"
SizeINTThe maximum size of the file this entry is allowed to match. -1 matches all files smaller than the maximum buffer size, 0 matches all files, and > 0 matches files up to that size. If a file is larger than the maximum buffer size, it will be partially buffered and the rest will be sent unprocessed.
PatternStringA regular expression pattern matching the area of text inside the file to modify; if this is left blank, and the host, file, or mimetype options aren't, this will be the last entry matched for sites matching the host, file, and mimetype.
ReplaceMultiline stringThe replacement text to use in place of the area of text matching the pattern; it may contain back references to strings captured using parenthesis in the pattern. A back reference to a captured string is in the form "$#", where # is a number from 1-9; "$0" will be replaced with the entire area of text matching the regular expression. Escape sequences may be used to represent unprintable characters, they are "\n" (newline), "\r" (carrier return), and "\t" (tab). To use a backslash as part of the replacement text, precede it with another backslash.
Applies toCheckboxes

This option is to select what the rewrite rule applies to; the options are:

  • Client header - rewrite the client header; this happens before Middleaman parses it so be careful not to remove any headers needed to handle the request properly. The Mimetype option serves no purpose for this.

  • Server header - rewrite the header from the remote web server; same conditions from client header apply.

  • Body - rewrite the body of the webpage or file.

  • POST data - rewrite POST/PUT data sent when submitting a form or uploading a file.

Cookies section

The cookies feature allows you to choose which hosts your browser is allowed to send and receive cookies to and from.

Table 5-23. Cookie global options

OptionTypeDescription
PolicyRadio buttonsThe action to take when no matching entry is found.

Table 5-24. Cookie entry options

OptionTypeDescription
Expiry year rangeTwo integersThe cookie expiry year range this entry applies to.
Expiry month rangeTwo stringsThe cookie expiry month range this entry applies to.
Expiry day rangeTwo integersThe cookie expiry day range this entry applies to.
Expiry weekday rangeTwo stringsThe cookie expiry weekday range this entry applies to.
Expiry hour rangeTwo integersThe cookie expiry hour range this entry applies to.
Expiry minute rangeTwo integersThe cookie expiry minute range this entry applies to.
DomainStringA regular expression matching the cookie's domain attribute this entry applies to.
PathStringA regular expression matching the cookie's path attribute this entry applies to.
DirectionRadio buttonsThe direction of the cookie this entry applies to; can be either in (Set-cookie sent by website), out (Cookie sent by browser), or both.
Time match modeRadio buttonsThe direction of the cookie this entry applies to; can be either in (Set-cookie sent by website), out (Cookie sent by browser), or both.

External section

The external feature allows you to use any program or script to parse the contents of a file.

Table 5-25. External entry options

OptionTypeDescription
ExecutableString

The path to the executable; if no absolute path is given, the path as given in the PATH environment variable is searched. Any number of arguments can be passed by separating them by spaces; if you're using a temporary file as the method to pass the contents of the file, it's path will be the last argument. When the program is executed, several environment variables are set to reflect the properties of the file being handled, they are:

  • HTTP_METHOD - Method used to request the file.

  • HTTP_HOST - Host HTTP request was made to.

  • HTTP_FILE - File HTTP request was made for.

  • HTTP_PORT - Port HTTP request was made to.

  • IP - IP address of client making request.

  • INTERFACE - IP address of the interface the client connected to.

  • PORT - Port the client connected to.

Additionally, for every header received from the remote website and set by a client, an environment variable is set. All the environment variables for the server's headers start with SERVER_, and the client's start with CLIENT_; All '-' (dashes) in the header type are converted to '_' (underscores), and all characters are in uppercase. If an executable returns with a non-zero status code, the original content is returned.

Mime typeStringA regular expression matching the MIME-type's this entry applies to, leave blank to match everything.
SizeINTThe maximum size of the file this entry is allowed to match. -1 matches all files smaller than the maximum buffer size, 0 matches all files, and > 0 matches files up to that size. If a file is larger than the maximum buffer size, it will be partially buffered and the rest will be sent unprocessed.
New mime typeStringThe MIME-type of the content returned from the external program, leave blank to have the original MIME-type preserved. If this is set to STDIN, the external program is expected to write "Content-type: <mimetype>" followed by 2 newlines as it's first output, where <mimetype> is the new MIME-type.
TypeRadio buttonsThe method which content is passed to the external program; if set to Pipe the content is piped to the program's STDIN, if set to File the content is stored in a temporary file and it's path is passed as the last argument.

Keyword section

The keyword filtering feature allows you to block pages which may contain inappropriate content using a scoring system. When the host, file, mimetype, and keyword in an entry matches a file, it's score is added to the total score; when that total score exceeds the threshold, the page is deemed inappropriate and blocked.

Table 5-26. Keyword global options

OptionTypeDescription
ThresholdINTThe number the total score must equal or exceed until it's blocked.
TemplateStringThe template to send when a page exceeds the threshold.

Table 5-27. Keyword entry options

OptionTypeDescription
Mime typeStringA regular expression matching the mimetype's this entry applied to; it is highly advisable that you set this to something, otherwise all file's will be checked; if you're unsure, set this to "text/"
SizeINTThe maximum size of the file this entry is allowed to match. -1 matches all files smaller than the maximum buffer size, 0 matches all files, and > 0 matches files up to that size. If a file is larger than the maximum buffer size, it will be partially buffered and the rest will be sent unprocessed.
KeywordStringA regular expression matching anything in the body of the document considered inappropriate, leave blank to match everything.
ScoreINTThe score this entry adds to the total score when it matches; this may be a positive or negative integer.

Limits section

The limits feature allows access to the proxy server to be restricted based on date/time, bandwidth usage, and requests made. More than one entry may be allowed to match, in this case the last to match is used; this makes it possible for example to limit bandwidth usage between Monday and Friday to 1GB, and not allow more than 200MB per day.

Table 5-28. Limits entry options

OptionTypeDescription
ActionRadio buttonsThe action to take when this entry matches. If deny, any request falling into the specified time range is blocked, otherwise the request is allowed. When limiting transfered data or requests is desired, an allow action should be used which will then become a deny action when the limit is reached.
TemplateStringThe template to send when this entry results in access being denied. The specified template is only sent if the page was blocked due to the time restrictions, 'maxbandwidth' and 'maxrequests' are used for excessive bandwidth or requests, respectively.
Month rangeTwo stringsThe range of months this entry applies to.
Day rangeTwo integersThe range of days this entry applies to.
Weekday rangeTwo stringsThe range of weekdays this entry applies to.
Hour rangeTwo integersThe hour range this entry applies to.
Minute rangeTwo integersThe range of minutes this entry applies to.
Byte transfer limitUINTThe maximum number of bytes that may be downloaded by the proxy during the time span this entry matches.
Current bytesUINTThe current number of bytes transferred in this time span; this setting is not saved, reloading the configuration or restarting the proxy will reset it.
Request limitUINTThe maximum number of requests that may be made by a client during the time span this entry matches.
Current requestsUINTThe current number of requests made in this time span; this setting is not saved, reloading the configuration or restarting the proxy will reset it.
Time match modeRadio buttonsThe method used to match the time. Absolute will match the exact date/time, All ranges will match if the current time falls into every choses range. For example, with absolute selected with the weekday and time ranges respectively set to Monday-Friday, 8:00 to 17:00, the entry will match any time between 8am Monday and 5pm Friday; with all ranges selected the entry will match 8am to 5pm on any day between Monday and Friday.
FlagsCheckboxesThe 'limit cache transfers' option will place bandwidth limits on transfers to the client that didn't result in any bandwidth from a website.