mpop 0.7.0

Table of Contents

Next: , Up: (dir)

mpop

This manual was last updated June 17, 2005 for version 0.7.0 of mpop.

Copyright (C) 2005 Martin Lambers

This program, including this manual, is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program, including this manual, is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.

Next: , Previous: Top, Up: Top

1 Introduction

mpop is a POP3 client.

In its main mode of operation, it retrieves mails from one or more POP3 mailboxes, does some optional filtering, and delivers them through a mail delivery agent (MDA). Mails that were successfully delivered before can be identified and ignored. Successfully delivered mails will be deleted from POP3 server by default.

The best way to start is probably to have a look at the Examples section. See Examples.

In addition to the mail retrieval mode, mpop can be used in server information mode. In this mode, mpop prints as much information as it can get about a given POP3 server (greeting, supported features, login delay, maximum mail size, ...).

Normally, a configuration file contains information about which POP3 server to use and how to use it, but almost all settings can also be configured on the command line.

POP3 server information is organized in accounts. Each account describes one POP3 server: host name, authentication settings, TLS settings, and so on. Each configuration file can define multiple accounts.

Supported features include:

Next: , Previous: Introduction, Up: Top

2 Configuration file

If it exists and is readable, a user configuration file will be loaded ($HOME/.mpoprc by default). This file must have no more permissions than user read/write. Configuration file settings can be changed by command line options.

A configuration file is a simple text file. Empty lines and comment lines (whose first non-blank character is '#') are ignored. Every other line must contain a command and may contain an argument to that command. The argument may be enclosed in double quotes (").

If the first character of a filename is the tilde (~), this tilde will be replaced by $HOME.

If a command accepts the argument on, it also accepts an empty argument and treats that the same as on.

Commands form groups. Each group starts with the account command and defines the settings for one POP3 server.

See Examples.

2.1 General commands

defaults
Set defaults. The following configuration commands will set default values for all following account definitions.
account name [ : account[,...] ]
Start a new account definition with the given name. The current default values are filled in (see defaults).
If a colon and a list of previously defined accounts is given after the account name, the new account, with the filled in default values, will inherit all settings from the accounts in the list.
host hostname
The POP3 server to retrieve mails from. The argument may be a host name or a network address. Every account definition must contain this command.
port number
The port that the POP3 server listens on. The default is 110, unless TLS without STARTTLS is used, in which case it is 995.
connect_timeout (off|seconds)
Set or unset the connect timeout, in seconds. The argument off means that no timeout will be set, which means that the operating system default will be used.
pipelining (on|off)
Enable or disable POP3 pipelining. The default if off, because that works with every POP3 server. Most servers support pipelining, and using it usually makes POP3 sessions much faster, so it is recommended to activate it. See Pipelining.

2.2 Authentication commands

See Authentication.

auth [(on|method)]
This command chooses the POP3 authentication method. With the argument on, mpop will choose the best one available for you (see below). This is the default. Accepted methods are user, apop, plain, cram-md5, digest-md5, gssapi, external, login, and ntlm. See Authentication.

user [username]
Set your user name for POP3 authentication. An empty argument unsets the user name.
password [secret]
Set your password for POP3 authentication. An empty argument unsets the password. If no password is set but one is needed during authentication, mpop will try to find it in ~/.netrc, and if that fails, mpop will prompt you for it. See Authentication.
ntlmdomain [ntlmdomain]
Set a domain for the ntlm authentication method. The default is to use no domain (equivalent to an empty argument), but some servers seem to require one, even if it is an arbitrary string.

2.3 TLS commands

See Transport Layer Security.

tls [(on|off)]
This command enables or disables TLS/SSL encrypted connections to the POP3 server. Not every server supports TLS, and many that do require the tls_starttls off command. See Transport Layer Security.
tls_trust_file [file]
This command activates strict server certificate verification. The file must contain one or more certificates of trusted Certification Authorities (CAs) in PEM format. An empty argument disables this feature. See Transport Layer Security.
tls_key_file [file]
This command (together with the tls_cert_file) command enables mpop to send a client certificate to the POP3 server if requested. The file must contain the private key of a certificate in PEM format. An empty argument disables this feature. See Transport Layer Security.
tls_cert_file [file]
This command (together with the tls_key_file command) enables mpop to send a client certificate to the POP3 server if requested. The file must contain a certificate in PEM format. An empty argument disables this feature. See Transport Layer Security.
tls_certcheck [(on|off)]
This command enables or disables sanity checks for the server certificate. These checks are enabled by default, but can cause difficulties. See Transport Layer Security.
tls_starttls [(on|off)]
This command enables or disables the use of the STARTTLS POP3 command to start TLS encryption. It is enabled by default. See Transport Layer Security.

2.4 Commands specific to mail retrieval mode

See Mail retrieval mode.

delivery method method_arguments...
How to deliver messages received from this account. If the delivery method needs to parse the mail headers for an envelope from address (the mda method if the command contains %F, and the mbox method), then it needs to create a temporary file to store the mail headers (but not the body) in. See $TMPDIR in Environment / Files.
uidls_file filename
The file to store UIDLs in. These are needed to identify new messages (see only_new). The file is only used for POP3 servers that support the UIDL command; see Server information mode. The default is ~/.mpop_uidls. One file can be shared by multiple accounts. An UIDLs file contains at most as much UIDs as there are mails on the server, for each account it is used for.
only_new [(on|off)]
If this option is off, mpop processes all messages (this is the default). If it is on, mpop processes only new messages and ignores already retrieved messages. To distinguish between new and retrieved messages, it is necessary that the POP3 server supports the POP3 UIDL command (see Server information mode). For each retrieved mail that is left on the server, mpop stores a unique ID in a special file (see uidls_file). When it runs the next time, it can recognize already retrieved mails because it knows their UIDs.
If the POP3 server does not support the UIDL command, all messages are new for mpop, so there's no difference if this option is on or off for these servers.
This command is normally used in combination with the keep command, see keep.
keep [(on|off)]
Keep all mails on the POP3 server, never delete them. The default behavior is to delete mails that have been successfully delivered or filtered by kill filters.
killsize (off|size)
Mails larger than the given size will be deleted, not downloaded (unless the keep command is used, in which case they will just be skipped). The size argument must be zero or greater. If it is followed by a 'k' or an 'm', the size is measured in kilobytes/megabytes instead of bytes. Note that some POP3 servers report slightly incorrect sizes for mails. See Filtering.
skipsize (off|size)
Mails larger than the given size will be skipped (not downloaded). The size argument must be zero or greater. If it is followed by a 'k' or an 'm', the size is measured in kilobytes/megabytes instead of bytes. Note that some POP3 servers report slightly incorrect sizes for mails. See Filtering.
filter [COMMAND]
Set a filter which will decide whether to retrieve, skip, or delete each mail by investigating the mail's headers. The POP3 server must support the POP3 TOP command for this to work; see Server information mode. An empty argument disables filtering.
All occurences of %F in the command will be replaced with the envelope from address of the current message (or MAILER-DAEMON if none is found). All occurences of %S in the command will be replaced with the size of the current mail as reported by the POP3 server.
The mail headers (plus the blank line separating the headers from the body) will be piped to the command. Based on the return code, mpop decides what to do with the mail: Return codes greater than or equal to 3 mean that an error occurred. The sysexits.h error codes may be used to give information about the kind of the error, but this is not necessary. See Filtering.

Next: , Previous: Configuration file, Up: Top

3 Invocation

3.1 Synopsis

3.2 Exit code

The standard exit codes from sysexits.h are used.

3.3 Environment / Files

$HOME/.mpoprc
The default user configuration file.
$HOME/.mpop_uidls
Default file to store UIDLs in.
$HOME/.netrc
The .netrc file contains login information. If a password is not found in the configuration file, msmtp will search it in .netrc before prompting the user for it. The syntax of .netrc is described in the netrc(5) or ftp(1) manual page.
$USER, $LOGNAME
These variables override the user's login name. $LOGNAME is only used if $USER is unset. The user's login name is used for Received headers.
$TMPDIR
Directory to create temporary files in. If this is unset, a system specific default directory is used.

3.4 Options

Options override configuration file settings. The following options are accepted:

3.4.1 General options

--version
Print version information. This includes information about the library used for TLS/SSL support (if any), the library used for authentication, and the authentication mechanisms supported by this library.
--help
Print help.
-P
--pretend
Print the configuration settings that would be used, but do not take further action. An asterisk ('*') will be printed instead of the password.
-d
--debug
Print lots of debugging information, including the whole conversation with the POP3 server. Be careful with this option: the (potentially dangerous) output will not be sanitized, and your password may get printed in an easily decodable format!
This option implies --quiet, because the debugging output would otherwise interfere with the normal output.

3.4.2 Changing the mode of operation

-S
--serverinfo
Print information about the POP3 server and exit. This includes information about supported features (authentication methods, TOP command, ...), about parameters (time for which mails will not be deleted, minimum time between logins, ...), and about the TLS certificate (if TLS is active). See Server information mode.

3.4.3 Configuration options

Most options in this category correspond to a configuration file command. Please refer to Configuration file for detailed information.

-C filename
--file=filename
Use the given file instead of $HOME/.mpoprc as the configuration file.
--host=hostname
Use this POP3 server with settings from the command line; do not use any configuration file data. This option disables loading of the configuration file. You cannot use both this option and account names on the command line.
--port=number
Set the port number to connect to. See port.
--connect-timeout=(off|seconds)
Set or unset the connect timeout, in seconds. See connect_timeout.
--pipelining=(on|off)
Enable or disable POP3 pipelining. See pipelining.
--auth[=(on|method)]
Set the authentication method to automatic (with "on") or manually choose an authentication method. See auth.
--user=[username]
Set or unset the user name for authentication. See user.
--tls[=(on|off)]
Enable or disable TLS. See tls.
--tls-trust-file=[file]
Set or unset a trust file for TLS encryption. See tls_trust_file.
--tls-key-file=[file]
Set or unset a key file for TLS encryption. See tls_key_file.
--tls-cert-file=[file]
Set or unset a cert file for TLS encryption. See tls_cert_file.
--tls-certcheck[=(on|off)]
Enable or disable server certificate checks for TLS encryption. See tls_certcheck.
--tls-starttls[=(on|off)]
Enable or disable STARTTLS for TLS encryption. See tls_starttls.

3.4.4 Options specific to mail retrieval mode

-q
--quiet
Do not print progress information.
-a
--auth-only
Authenticate only; do not retrieve mail. Useful for SMTP-after-POP.
-s
--status-only
Print number and size of mails in each account only; do not retrieve mail.
-n
--only-new[=(on|off)]
Process only new messages, ignore already retrieved messages. See only_new.
-k
--keep[=(on|off)]
Do not delete mails from POP3 servers, regardless of other options or settings. See keep.
--killsize=(off|size)
Set or unset kill size. See killsize.
--skipsize=(off|size)
Set or unset skip size. See skipsize.
--filter=[command]
Set a filter which will decide whether to retrieve, skip, or delete each mail by investigating the mail's headers. See filter.
--delivery=method,method_arguments...
How to deliver messages received from this account. See delivery. Note that a comma is used instead of a blank to separate the method from its arguments.
--uidls-file=filename
File to store UIDLs in. See uidls_file.

Next: , Previous: Invocation, Up: Top

4 POP3 features

Next: , Up: POP3 features

4.1 Transport Layer Security

Quoting from RFC2246 - the TLS 1.0 protocol specification:
"The TLS protocol provides communications privacy over the Internet. The protocol allows client/server applications to communicate in a way that is designed to prevent eavesdropping, tampering, or message forgery."

POP3 servers can use TLS in one of two modes:

mpop can switch between these modes with the tls_starttls command (see tls_starttls) command or the --tls-starttls option (see –tls-starttls).

When TLS is started, the server sends a certificate to identify itself. This certificate contains information about the certificate owner, the certificate issuer, and the activation and expiration times of the certificate. This information can be displayed in server information mode. See Server information mode.

Some sanity checks are done with the server certificate. These include:

Sometimes one of these checks fail. mpop will abort the connection in this case. If the user still wants to use this POP3 server with TLS, the sanity checks can be switched off with tls_certcheck or --tls-certcheck (see tls_certcheck, –tls-certcheck).

Note that the POP3 server cannot be fully trusted just because the certificate passes the sanity checks. To verify that the user can trust the POP3 server, it is necessary to use a (list of) certificates of Certification Authorities (CAs) that are trusted. If mpop can verify that the server certificate was issued by one of these CAs, then the POP3 server is trusted. A file containing CA certificates can be set with tls_trust_file or --tls-trust-file (see tls_trust_file, –tls-trust-file).

If the server requests it, the client can send a certificate, too. This allows the server to verify the identity of the client. See the EXTERNAL mechanism in Authentication. The tls_key_file/tls_cert_file commands or the --tls-key-file/--tls-cert-file options can be used to set a client certificate. See tls_key_file/–tls-key-file, tls_cert_file/–tls-cert-file. Note that GnuTLS will only send a client certificate if it matches one of the CAs advertised by the server. If you set a client certificate but it is not send to the server, probably does not match any CA that the server trusts.

Next: , Previous: Transport Layer Security, Up: POP3 features

4.2 Authentication

POP3 servers require a client to authenticate itself before it is allowed to retrieve mail.

Multiple authentication methods exist. Most POP3 servers support only some of them. Some methods send authentication data in plain text (or nearly plain text) to the server. These methods should only be used when TLS is active to prevent others from stealing the password. See Transport Layer Security.

mpop supports a subset of the following authentication methods:

It depends on the underlying authentication library and its version whether a particular method is supported or not. Use the --version to find out which methods are supported by your version of mpop.

Authentication data can be set with the user and password commands or with the --user option. See user, password, –user. If no password is set but one is needed during authentication, mpop will try to find it in ~/.netrc, and if that fails, mpop will prompt you for it.

The authentication method can be chosen with the auth command or --auth option, but it is usually sufficient to just use the on argument to let mpop choose the method itself. See auth, –auth.

If mpop chooses the method itself, it will not choose a method that sends plain text authentication data when TLS is not active. This means that only APOP, CRAM-MD5, DIGEST-MD5, GSSAPI, and NTLM are available when TLS is inactive. USER, PLAIN and LOGIN are only available when TLS is active. If you really want to send clear text authentication data, you have to force mpop to do that by setting the authentication method to USER, PLAIN or LOGIN while TLS is off.

Previous: Authentication, Up: POP3 features

4.3 Pipelining

Pipelining can be enabled with the pipelining command or the --pipelining option. In short: enable it, it will speed up the POP3 sessions.

A POP3 client that sends multiple POP3 commands at once to a POP3 server before starting to read the server's answers is using POP3 pipelining. Since the client does not have to wait for the server's answer before sending the next command, and the server does not have to wait for the next command from the client, pipelining can speed up a POP3 session substantially.

Most POP3 servers support pipelining. Those that support the CAPA command, too, will advertize the pipelining capability; see Server information mode. But even if a server does not support CAPA, it probably still supports pipelining.

Pipelining in mpop works by sending up to PIPELINE_MAX commands to the server, then begin to read its answers, and refill the command pipeline when the number of unanswered commands drops to PIPELINE_MIN. PIPELINE_MIN and PIPELINE_MAX are compile time constants.

See pipelining, –pipelining.

Next: , Previous: POP3 features, Up: Top

5 Mail retrieval mode

In this mode, mpop retrieves mail from one or more POP3 servers. It delivers each of them using the method that was given with the delivery command or --delivery option. See delivery, –delivery.

While retrieving the mail, mpop displays approximate progress information, which can be turned off with the --quiet option; see –quiet.

If the delivery succeeded, the mail is deleted from the POP3 server by default. The keep command and --keep option can prevent the deletion of mails; see keep, –keep.

Important note: Some POP3 servers will delete mails without any user interaction. See EXPIRE in Server information mode. mpop can do nothing about that.

If you don't want to download certain mails, but skip them or delete them directly, you can do filtering based on the mail headers. See Filtering.

If you just want to know if you have new mails (and if so, how many), use the --status-only option. See –status-only.

If you just want to authenticate to the POP3 server, but don't want to look at your mails, use the --auth-only option. See –auth-only. This can be useful for sending mail through SMTP servers that require SMTP-after-POP (aka POP-before-SMTP).

Before mpop delivers a mail, it prepends a Received header to it. This is necessary if the delivery method transmits the mail to an SMTP server, for example. mpop does not change the contents of the mail in any other way.

Next: , Previous: Filtering, Up: Top

6 Server information mode

In server information mode, mpop prints as much information about the POP3 server as it can get and then exits.

The POP3 features that can be detected are:

If TLS is activated for server information mode, the following information will be printed about the POP3 server's TLS certificate (if available):

Next: , Previous: Mail retrieval mode, Up: Top

7 Filtering

There are three filtering commands available. They will be executed in the following order:

  1. killsize
  2. skipsize
  3. filter

If a filtering command applies to a mail, the remaining filters will not be executed.

The POP3 server must support the POP3 TOP command (Server information mode) for filtering with a filter command: It is used to read the mail headers (plus the blank line separating the header from the body) and pipe them to the filter command.

Note that, if the filter decides that the mail should be retrieved, the complete mail has to be downloaded, including the headers, so the headers will be downloaded twice. This is because there's no way in POP3 to download just the mail body. Sometimes this overhead surpasses the savings of the filtering.

The filter command looks at the mail headers and signals with its exit code what mpop should do with the mail:

Return codes greater than or equal to 3 mean that an error occurred. The sysexits.h error codes may be used to give information about the kind of the error, but this is optional.

Since the filter command will be passed to a shell, you can use all shell command constructs in addition to just calling a script or program. This allows flexible filter constructs. See Filtering with SpamAssassin.

Some POP3 servers count end-of-line characters as two bytes (CRLF) instead of one (LF), so that the size of a mail as reported by the POP3 server is slightly larger than the actual size. The filters use the size values reported by the POP3 server since they cannot know the actual size in advance. Thus you cannot rely on exact size filtering.

Next: , Previous: Server information mode, Up: Top

8 Examples

Next: , Up: Examples

8.1 A configuration file

     # Default values for all accounts
     defaults
     pipelining on
     delivery mda "/usr/bin/procmail -f %F -d $USER"
     #delivery mda "/usr/sbin/sendmail -oi -oem -f %F -- $USER"
     #delivery maildir ~/Mail/incoming
     #delivery mbox ~/Mail/new
     #delivery simple_mbox ~/Mail/new
     tls on
     tls_trust_file ~/.ca-bundle.pem
     
     # My provider, where I have two pop3 mailboxes
     account provider1
     host mx.provider.example
     user joe_smith
     password secret
     tls_starttls off
     
     account provider2 : provider1
     user joey
     password secret2
     
     # A freemail service
     account freemail
     host pop.freemail.example
     user 123
     password pass
     
     # Be sure to never loose mail while testing a new
     # filter script for the freemail service
     account freemail-test : freemail
     filter ~/bin/experimental-filter.sh --from=%F --size=%S
     keep on
     
     # Set a default account
     account default : provider1

Previous: A configuration file, Up: Examples

8.2 Filtering with SpamAssassin

Use the following to delete all mails that SpamAssassin classifies as spam: 

     filter "/path/to/spamc -c > /dev/null"

Since no message body is passed to SpamAssassin, you should disable all body-specific tests in the SpamAssassin configuration file; for example set use_bayes 0.

If your mail provider runs SpamAssassin for you, you just have to check for the result. The following script can do that when used as an mpop filter: 

     #!/bin/sh
     if [ "`grep "^X-Spam-Status: Yes"`" ]; then
         exit 1  # kill this message
     else
         exit 0  # proceed normally
     fi

Since the filter command is passed to a shell, all shell constructs are usable, so you can also use this directly: 

     filter if [ "`grep "^X-Spam-Status: Yes"`" ]; then exit 1; else exit 0; fi

Previous: Examples, Up: Top

9 Development

The homepage of mpop is http://mpop.sourceforge.net/; the SourceForge project page is at http://sourceforge.net/projects/mpop/.

Please send any questions, suggestions, and bug reports to Martin Lambers (marlam@marlam.de, OpenPGP key: http://www.marlam.de/key.txt). If you send a bug report, please include the output of mpop --version.