Your IP : 18.191.241.222
<!DOCTYPE html>
<html>
<head>
<title>ProFTPD module mod_digest</title>
</head>
<body bgcolor=white>
<hr>
<center>
<h2><b>ProFTPD module <code>mod_digest</code></b></h2>
</center>
<hr><br>
<p>
The <code>mod_digest</code> module offers functionality for calculating the hash
(or <em>digest</em>) value of files. This is particularly useful when verifying
the integrity of files. This functionality is used by the following custom
FTP commands:
<ul>
<li><code>XCRC</code> (requests CRC32 digest/checksum)
<li><code>MD5/XMD5</code> (requests MD5 digest/checksum)
<li><code>XSHA</code>/<code>XSHA1</code> (requests SHA1 digest/checksum)
<li><code>XSHA256</code> (requests SHA256 digest/checksum)
<li><code>XSHA512</code> (requests SHA512 digest/checksum)
</ul>
In addition, <code>mod_digest</code> supports the more modern <a href="https://tools.ietf.org/html/draft-bryan-ftpext-hash-02"><code>HASH</code></a> command.
<p>
Depending on the file size and the hash function, it takes a fair amount of
CPU and IO resources to calculate the result. Therefore decide wisely where
to enable the features and set the <a href="#DigestMaxSize">DigestMaxSize</a>
configuration directive appropriately.
<p>
This module was compiled and tested against ProFTPD 1.3.3 Installation
instructions are discussed <a href="#Installation">here</a>.
<p>
The most current version of <code>mod_digest</code> is distributed with the
ProFTPD source code.
<h2>Author</h2>
<p>
Please contact TJ Saunders <tj <i>at</i> castaglia.org> with any
questions, concerns, or suggestions regarding this module.
<h2>Thanks</h2>
<p>
<i>2016-01-09</i>: Thanks to Mathias Berchtold <mb <i>at</i>
smartftp.com> for his original <code>mod_digest</code>, upon which this
version is based.
<h2>Directives</h2>
<ul>
<li><a href="#DigestAlgorithms">DigestAlgorithms</a>
<li><a href="#DigestCache">DigestCache</a>
<li><a href="#DigestDefaultAlgorithm">DigestDefaultAlgorithm</a>
<li><a href="#DigestEnable">DigestEnable</a>
<li><a href="#DigestEngine">DigestEngine</a>
<li><a href="#DigestMaxSize">DigestMaxSize</a>
<li><a href="#DigestOptions">DigestOptions</a>
</ul>
<hr>
<h3><a name="DigestAlgorithms">DigestAlgorithms</a></h3>
<strong>Syntax:</strong> DigestAlgorithms <em>["crc32"|"md5"|"sha1"|"sha256"|"sha512"|"all"]</em><br>
<strong>Default:</strong> DigestAlgorithms all<br>
<strong>Context:</strong> server config, <VirtualHost>, <Global>, <Anonymous><br>
<strong>Module:</strong> mod_digest<br>
<strong>Compatibility:</strong> 1.3.6rc2 or later
<p>
The <code>DigestAlgorithms</code> directive configures the enabled digest
algorithms. If no <code>DigestAlgorithms</code> directive is configured, then
<b>all</b> supported digest algorithms are enabled.
<p>
Enabled digest algorithms are announced/discovered via the <code>FEAT</code>
response.
The following algorithms are currently supported by <code>mod_digest</code>:
<ul>
<li><code>crc32</code> (<i>e.g.</i> for the <code>XCRC</code> command)
<li><code>md5</code> (<i>e.g.</i> for the <code>XMD5</code> command)
<li><code>sha1</code> (<i>e.g.</i> for the <code>XSHA</code>/<code>XSHA1</code> commands)
<li><code>sha256</code> (<i>e.g.</i> for the <code>XSHA256</code> command)
<li><code>sha512</code> (<i>e.g.</i> for the <code>XSHA512</code> command)
</ul>
<p>
<hr>
<h3><a name="DigestCache">DigestCache</a></h3>
<strong>Syntax:</strong> DigestCache <em>on|off|"size" count ["maxAge" secs]</em><br>
<strong>Default:</strong> DigestCache size 10000 maxAge 30s<br>
<strong>Context:</strong> server config, <VirtualHost>, <Global>, <Anonymous><br>
<strong>Module:</strong> mod_digest<br>
<strong>Compatibility:</strong> 1.3.6rc2 or later
<p>
The <code>mod_digest</code> module will cache the results of any checksum
command, on a per-file basis. This improves performance, and reduces
computational overhead. To disable this caching for any reason, use this
directive:
<pre>
# Disable checksum caching
DigestCache off
</pre>
<b>This is not recommended.</b>
<p>
The <code>DigestCache</code> directive can also be used to configure/tune the
<em>max-size</em> of the in-memory cache. Note that once the maximum cache
size is reached, any checksum FTP commands will be temporarily refused:
<pre>
# Use a smaller cache size
DigestCache size 100
</pre>
Cached digests will be expired/ignored after 30 seconds, by default. To change
the expiration, you would use:
<pre>
# Retain cached entries longer
DigestCache maxAge 60s
</pre>
<p>
If <em>on</em> is used, <code>mod_digest</code> will use the default
<em>max-size</em> of 10000:
<pre>
DigestCache on
</pre>
<p>
<hr>
<h3><a name="DigestDefaultAlgorithm">DigestDefaultAlgorithm</a></h3>
<strong>Syntax:</strong> DigestDefaultAlgorithm <em>algo</em><br>
<strong>Default:</strong> DigestDefaultAlgorithm sha1<br>
<strong>Context:</strong> server config, <VirtualHost>, <Global><br>
<strong>Module:</strong> mod_digest<br>
<strong>Compatibility:</strong> 1.3.6rc3 or later
<p>
The default digest algorithm that the <code>mod_digest</code> module uses,
for <i>e.g.</i> opportunistic digesting of file transfers, is SHA1. For
selecting a different default algorithm, use the
<code>DigestDefaultAlgorithm</code> directive:
<pre>
# Use MD5 rather than SHA1 as the default algorithm
DigestDefaultAlgorithm md5
</pre>
<p>
<b>Note</b> that the <code>DigestAlgorithms</code> directive takes precedence;
if the <code>DigestDefaultAlgorithm</code> is not included in the
<code>DigestAlgorithms</code>, the default algorithm setting will be ignored.
<p>
<hr>
<h3><a name="DigestEnable">DigestEnable</a></h3>
<strong>Syntax:</strong> DigestEnable <em>on|off</em><br>
<strong>Default:</strong> Non<br>
<strong>Context:</strong> <code><Directory></code>, <code>.ftpaccess</code><br>
<strong>Module:</strong> mod_digest<br>
<strong>Compatibility:</strong> 1.3.6rc2 or later
<p>
The <code>DigestEnable</code> directive can be used to block or prevent
checksumming/digests on files in the configured <code><Directory></code>.
This can be <b>very</b> useful for preventing checksumming of files located
on network-mounted filesystems, for example.
<p>
<hr>
<h3><a name="DigestEngine">DigestEngine</a></h3>
<strong>Syntax:</strong> DigestEngine <em>on|off</em><br>
<strong>Default:</strong> DigestEngine on<br>
<strong>Context:</strong> server config, <VirtualHost>, <Global>, <Anonymous><br>
<strong>Module:</strong> mod_digest<br>
<strong>Compatibility:</strong> 1.3.6rc2 or later
<p>
The <code>DigestEngine</code> directive enables or disables the handling of
the checksum-related FTP commands by <code>mod_digest</code>, <i>i.e.</i>:
<ul>
<li><code>XCRC</code>
<li><code>XMD5</code>
<li><code>XSHA</code>
<li><code>XSHA1</code>
<li><code>XSHA256</code>
<li><code>XSHA512</code>
</ul>
If the parameter is <em>off</em>, then these commands will be ignored.
<p>
<hr>
<h3><a name="DigestMaxSize">DigestMaxSize</a></h3>
<strong>Syntax:</strong> DigestMaxSize <em>number [units]</em><br>
<strong>Default:</strong> None<br>
<strong>Context:</strong> server config, <VirtualHost>, <Global>, <Anonymous><br>
<strong>Module:</strong> mod_digest<br>
<strong>Compatibility:</strong> 1.3.6rc2 or later
<p>
The <code>DigestMaxSize</code> directive configures the maximum number of bytes
a single hash command is allowed to read from a file. If the number of bytes
to be read from the file is greater than the configured <em>number</em> the
server will refuse that command.
<p>
If no <code>DigestMaxSize</code> directive is configured, then there is no
limit. It is highly <b>recommended</b> to set an upper limit.
<p>
Example:
<pre>
# Limit hashing to 1GB of data
DigestMaxSize 1 GB
</pre>
<p>
<hr>
<h3><a name="DigestOptions">DigestOptions</a></h3>
<strong>Syntax:</strong> DigestOptions <em>opt1 ...</em><br>
<strong>Default:</strong> None<br>
<strong>Context:</strong> server config, <code><VirtualHost></code>, <code><Global></code><br>
<strong>Module:</strong> mod_digest<br>
<strong>Compatibility:</strong> 1.3.6rc2 and later
<p>
The <code>DigestOptions</code> directive is used to configure various optional
behavior of <code>mod_digest</code>.
<p>
The currently implemented options are:
<ul>
<li><code>NoTransferCache</code><br>
<p>
The <code>mod_digest</code> module will automatically calculate <b>and</b>
cache the results of any transferred file, on a per-file basis. This is
done assuming that many FTP clients will want to verify the integrity of
the file just uploaded/downloaded. This improves performance, and
reduces computational overhead. To disable this caching for any reason,
use this option. <b>Not recommended.</b>
<p>
<b>Note</b>: The <code>NoTransferCache</code> option is
<em>automatically</em> enabled when using ProFTPD versions before
1.3.6rc2, due to bugs/missing support in the older versions.
</li>
</ul>
<p>
<hr>
<h2><a name="Installation">Installation</a></h2>
The <code>mod_digest</code> module is distributed with ProFTPD. Follow the
normal steps for using third-party modules in ProFTPD:
<pre>
$ ./configure --enable-openssl --with-modules=mod_digest
</pre>
To build <code>mod_digest</code> as a shared/DSO module:
<pre>
$ ./configure --enable-dso --enable-openssl --with-shared=mod_digest
</pre>
Then follow the usual steps:
<pre>
$ make
$ make install
</pre>
<p>
Alternatively, if your proftpd was compiled with DSO support, you can
use the <code>prxs</code> tool to build <code>mod_digest</code> as a shared
module:
<pre>
$ prxs -c -i -d mod_digest.c
</pre>
<p>
<hr>
<h2>Usage</h2>
Example Configuration
<pre>
<IfModule mod_digest.c>
# Set a limit on file sizes that can be digested
DigestMaxSize 1 GB
</IfModule>
</pre>
<p>
<b>Recording Uploaded/Downloaded File Checksums</b><br>
One particular use case that comes up is whether the <code>mod_digest</code>
can be used to record the digests ("checksums") of uploaded/downloaded files
in <i>e.g.</i> a SQL database. The answer is "yes", with some caveats.
<p>
First, here is a configuration excerpt showing show such functionality might
be implemented, using <code>mod_digest</code> and <code>mod_sql</code>:
<pre>
<IfModule mod_digest.c>
</IfModule>
<IfModule mod_sql.c>
...
SQLNamedQuery log-file-checksum FREEFORM "INSERT INTO file_checksums (user, file, algo, checksum) VALUES ('%u', '%f', '%{note:mod_digest.algo}', '%{note:mod_digest.digest}')"
SQLLog RETR,STOR log-file-checksum
...
</IfModule>
</pre>
As you can see, this makes use of the <code>%{note:...}</code> syntax of
the <code>SQLLog</code> directive; the same syntax <em>also</em> works for
<code>LogFormat</code> definitions as well. The <code>mod_digest</code> module
uses the following notes:
<ul>
<li><em>mod_digest.algo</em>
<p>
Name of the digest algorithm used, <i>e.g.</i> "SHA1".
</li>
<p>
<li><em>mod_digest.digest</em>
<p>
Calculated digest of the file as a hex-encoded lowercase string.
</li>
</ul>
<p>
Now, the caveats with this technique:
<ul>
<li>Does <b>not</b> work if the <code>NoTransferCache</code> <a href="#DigestOption">DigestOption</a> is used.
<li>Only works for binary, not ASCII, FTP uploads/downloads currently.
<li>Only works for uploads (<code>STOR</code>) and downloads (<code>RETR</code>), but not for appends (<code>APPE</code>) <b>or</b> resumed uploads/downloads (<code>REST</code> + <code>RETR/STOR</code>).
<li>Does <b>not</b> work for FTP downloads if <code>UseSendfile</code> is in effect.
</ul>
In addition, the order in which the <code>mod_digest</code> and
<code>mod_sql</code> appear in your build command is important;
<code>mod_digest</code> <em>must come <b>after</b></em> <code>mod_sql</code>,
otherwise the note values will <b>not</b> be populated properly in the
<code>SQLLog</code> statement. Thus, if you are building static modules,
your <code>--with-modules</code> parameter would look something like:
<pre>
$ ./configure --with-modules=mod_sql:mod_sql_mysql:mod_digest ...
</pre>
Or, if you are using shared modules, then your <code>LoadModule</code>
directives must look like:
<pre>
LoadModule mod_sql.c
LoadModule mod_sql_mysql.c
LoadModule mod_digest.c
</pre>
<!--
Why?
TCP-level checksums
packet-level checksums
_file_-level checksums (which is really what most people usually have in mind)
transfers interrupted by timeouts
SFTP has different ways of achieving this, via extensions (link to mod_sftp
docs on extensions)
validating uploads AND downloads (did I download everything? Did the upload
succeed?)
<p>
It's also recommended to disable all features within the <Anonymous> context. How?
<Anonymous>
<IfModule mod_digest.c>
DigestEngine off
</IfModule>
</Anonymous>
<p>
<b>Supported FTP Commands</b><br>
cmd path
cmd path [end]
cmd path [off] [len]
<pre>
XCRC "/path/to/file with spaces" 0 100
</pre>
-->
<p>
<hr>
<font size=2><b><i>
© Copyright 2016 TJ Saunders<br>
All Rights Reserved<br>
</i></b></font>
<hr>
</body>
</html>