pop3/imap servers for enterprises and isps - dovecot … · 0 0 peer heinlein dovecot pop3/imap...
TRANSCRIPT
Copy
right
© O
pen
Sour
ce P
ressPeer Heinlein
Dovecot
POP3/IMAP servers for enterprises and ISPs
Open Source Press
Copy
right
© O
pen
Sour
ce P
ress
The information in this book ist distributed on As is basis, without warranty. While every precau-tion has been taken in the preparation of this work, neither the authors nor the editors nor OpenSource Press GmbH shall have any liability to any person or entity with respect to any loss ordamage caused or alleged to be caused directly or indirectly by the information contained in it.Open Source Press and the Open Source Press logo are registered trade marks of Open SourcePress GmbH. Other product or company names mentioned herein may be the trademarks of theirrespective owners. Rather than use a trademark symbol with every occurence of a trademarkedname, we are using the names only in an editorial fashion and to the benefit of the trademarkowner, with no intention of infringement of the trademark.Original German edition © 2014 Open Source Press, MunichPeer Heinlein: Dovecot - POP3/IMAP-Server für Unternehmen und ISPs
Bibliographic information published by the Deutsche Nationalbibliothek
The Deutsche Nationalbibliothek lists this publication in the Deutsche Nationalbibliografie; de-tailed bibliographic data are available in the Internet at http://dnb.dnb.de.
Copyright © 2014 Open Source Press GmbHEditor: Markus WirtzTranslator: Katherine SalterTypesetting: textovia web application (http://textovia.com)Graphic Designer: Olga Saborov
ISBN: 9783955391072 (printed edition) http://www.opensourcepress.de
Copy
right
© O
pen
Sour
ce P
ressTable of Contents
Foreword by Timo Sirainen 13
Preface 15
1 Protocols and Terms 19
1.1 An Overview of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.2 What makes IMAP so complex? . . . . . . . . . . . . . . . . . . . . . . . . . . 21
I Dovecot for all purposes 23
2 Installation 25
2.1 Installation under Debian/Ubuntu . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.2 Installation under openSUSE/SLES . . . . . . . . . . . . . . . . . . . . . . . . 28
2.3 Installation under CentOS/RHEL . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.4 Starting Dovecot for the first time. . . . . . . . . . . . . . . . . . . . . . . . 29
2.5 The first login. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3 POP3 and IMAP at protocol level 33
3.1 POP3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.1.1 Test-session . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.2 IMAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
3.2.1 Design of the IMAP-protocol . . . . . . . . . . . . . . . . . . . . . . 37
3.2.2 Description of an IMAP session . . . . . . . . . . . . . . . . . . . . 39
3.2.3 IMAP in practical terms. . . . . . . . . . . . . . . . . . . . . . . . . . . 41
3
Copy
right
© O
pen
Sour
ce P
ress
3.2.4 Subscribing to IMAP folders . . . . . . . . . . . . . . . . . . . . . . . 49
4 Introduction to the configuration 51
4.1 The doveconf tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
4.2 Where and how to store your local configuration . . . . . . . . . . . . 54
4.3 Activating log messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.4 Specifying the IMAP namespace. . . . . . . . . . . . . . . . . . . . . . . . . . 56
4.4.1 The right hierarchy separator . . . . . . . . . . . . . . . . . . . . . . 57
4.4.2 Choosing the folder prefix. . . . . . . . . . . . . . . . . . . . . . . . . 59
4.4.3 Standardized names for trash, sent & Co. . . . . . . . . . . . . 61
5 Authentication 63
5.1 Basic settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
5.2 Authentication sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.3 /etc/passwd & /etc/shadow . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.4 passwd-file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68
5.5 LDAP queries/active directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5.5.1 To begin with: LDAP analysis with ldapsearch . . . . . . . . . 71
5.5.2 Configuration of password lookups (OpenLDAP). . . . . . . . 72
5.5.3 Configuration of authenticated binds (Active Directory,OpenLDAP). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
5.5.4 Setting LDAP search filters for userdb and passdbrequests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
5.6 SQL databases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
5.7 Performance optimization: prefetching with LDAP or SQL . . . . . . 80
5.8 Different or identical UIDs?. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
5.9 How not (!) to assign UID, GID and HOME . . . . . . . . . . . . . . . . . 83
5.10 Saving passwords: plain text or hash? . . . . . . . . . . . . . . . . . . . . 85
5.10.1 Authentication methods PLAIN/LOGIN . . . . . . . . . . . . . . . 85
5.10.2 Authentication methods CRAM/DIGEST . . . . . . . . . . . . . . 86
5.10.3 Setting up the CRAM process in Dovecot. . . . . . . . . . . . . 89
Table of Contents
4
Copy
right
© O
pen
Sour
ce P
ress
5.10.4 PLAIN/LOGIN only secure via SSL/TLS . . . . . . . . . . . . . . . . 89
5.10.5 How password hashes are saved. . . . . . . . . . . . . . . . . . . . 90
5.10.6 Login name or email address as login?. . . . . . . . . . . . . . . 91
5.10.7 Practical suggestion: different userdb and passdbqueries! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.11 Master user . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
5.12 Some extras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.12.1 Passdb extra fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
5.12.2 Userdb extra fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
5.13 A list of all users – the iterate query . . . . . . . . . . . . . . . . . . . . 101
6 How to configure Postfix as a relay in front of Dovecot 105
6.1 Delivery via dovecot-lda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
6.2 Delivery via Dovecot via LMTP . . . . . . . . . . . . . . . . . . . . . . . . . . 109
6.3 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
6.4 What recipients are there? Dynamic recipient verification . . . . 115
6.5 Setting up SMTP-Auth in Postfix . . . . . . . . . . . . . . . . . . . . . . . . 118
6.6 Less is more: do not go via Postfix . . . . . . . . . . . . . . . . . . . . . . 121
6.7 Advanced information on the configuration of Postfix . . . . . . . 124
7 mbox, Maildir and mdbox: a comparison of storage formats 125
7.1 Overview of the three formats . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.2 Maildir as an email storage format . . . . . . . . . . . . . . . . . . . . . . 127
7.2.1 Technical structure of Maildir . . . . . . . . . . . . . . . . . . . . . 128
7.2.2 File names of emails . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
7.2.3 Keywords: custom IMAP flags . . . . . . . . . . . . . . . . . . . . . 135
7.3 mdbox: the new format for larger setups . . . . . . . . . . . . . . . . . 137
7.3.1 Why mdbox is faster . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.3.2 Configuration of mdbox. . . . . . . . . . . . . . . . . . . . . . . . . . 139
7.3.3 Making space: doveadm purge . . . . . . . . . . . . . . . . . . . . 140
7.3.4 ALT storage: fast and slow data storage combined. . . . . 142
Table of Contents
5
Copy
right
© O
pen
Sour
ce P
ress
7.4 zlib compression on the fly: faster and more space-saving . . . . 144
7.5 auto: migration of storage formats during live operation . . . . . 147
7.5.1 auto: parallel operation of multiple storage formats . . . 147
7.5.2 A solid migration script as an example . . . . . . . . . . . . . 149
8 Partitions, file systems and downtimes during the file systemcheck 153
8.1 /srv/mail as data partition. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
8.1.1 Email data in a separate subdirectory . . . . . . . . . . . . . . 154
8.1.2 Email data on a separate data partition . . . . . . . . . . . . . 154
8.1.3 doveadm mount: in case something is missing . . . . . . . 156
8.2 Choosing the right file system . . . . . . . . . . . . . . . . . . . . . . . . . . 157
8.3 Measuring performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
8.4 Performance tuning the file system . . . . . . . . . . . . . . . . . . . . . . 160
8.4.1 atime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
8.4.2 The journal mode in ext3/ext4 . . . . . . . . . . . . . . . . . . . . 161
8.4.3 Optimized fstab entries . . . . . . . . . . . . . . . . . . . . . . . . . . 163
8.5 Out of service thanks to fsck . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
II Advanced Dovecot Installation 167
9 The IMAP namespace and shared folders 169
9.1 Necessary preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
9.2 Definition of a shared namespace . . . . . . . . . . . . . . . . . . . . . . . 173
9.3 The right hierarchy separator for a shared namespace . . . . . . . 174
9.4 Shared folders in mdbox or the auto:-mode . . . . . . . . . . . . . . . 176
9.5 Folders parallel to the INBOX . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
9.6 Public folders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
9.6.1 Managing the public namespace with a dummy user. . . 179
9.6.2 Display in the folder listing. . . . . . . . . . . . . . . . . . . . . . . 180
Table of Contents
6
Copy
right
© O
pen
Sour
ce P
ress
9.7 User-specific \Seen flags in shared folders and in the publicnamespace. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
10 Setting up SSL and TLS 183
10.1 How SSL/TLS works. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185
10.2 How to generate a self-signed key . . . . . . . . . . . . . . . . . . . . . . 187
10.3 Different keys for different IPs . . . . . . . . . . . . . . . . . . . . . . . . . 189
10.3.1 Different keys for different services . . . . . . . . . . . . . . . . 189
10.3.2 Different keys on different IPs . . . . . . . . . . . . . . . . . . . . 190
10.3.3 Server Name Indication (SNI) . . . . . . . . . . . . . . . . . . . . . 190
10.4 SSL/TLS and authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
11 Quotas 193
11.1 Structure of a quota system . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
11.2 Quota backends. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195
11.3 Activating the quota plugin . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
11.4 Configuring quota roots and the quota backend . . . . . . . . . . . 196
11.5 Definition of the quota rules . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
11.6 How to create quota warnings for users. . . . . . . . . . . . . . . . . . 200
11.7 Individual quota messages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
11.8 User-specific quotas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
11.8.1 passwd file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
11.8.2 LDAP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
11.8.3 MySQL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
11.8.4 PostgreSQL or SQLite . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203
11.9 Multiple quota roots. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
11.10 Working with quotas: doveadm quota . . . . . . . . . . . . . . . . . . . 204
11.11 Introducing quotas in large systems . . . . . . . . . . . . . . . . . . . . . 205
11.11.1 Medium-sized systems with no more than 500 users . . 205
11.11.2 Large systems with more than 500 users. . . . . . . . . . . . 206
11.12 The Quota policy server for Postfix . . . . . . . . . . . . . . . . . . . . . . 207
Table of Contents
7
Copy
right
© O
pen
Sour
ce P
ress
12 Server-side email filtering with Sieve 211
12.1 Setting up Sieve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
12.2 Out of office response via Sieve script . . . . . . . . . . . . . . . . . . . 216
12.3 Converting old procmail scripts . . . . . . . . . . . . . . . . . . . . . . . . . 219
12.4 Applying Sieve scripts to emails that have already beenreceived . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219
13 Email extensions 223
13.1 The recipient delimiter extends email addresses . . . . . . . . . . . . 223
13.2 Automatic saving of emails in IMAP folders . . . . . . . . . . . . . . . 224
14 Daily maintenance of the email storage with doveadm 227
14.1 General operation of doveadm. . . . . . . . . . . . . . . . . . . . . . . . . . 228
14.2 Managing IMAP folders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
14.2.1 Creating, renaming and deleting folders. . . . . . . . . . . . . 230
14.2.2 Converting special characters in folder names . . . . . . . . 230
14.2.3 Querying the status and number of emails in a folder . . 231
14.2.4 Subscribing to folders . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
14.3 Searching for emails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
14.3.1 Directory of all possible search keys . . . . . . . . . . . . . . . . 234
14.3.2 Dates in search criteria . . . . . . . . . . . . . . . . . . . . . . . . . . 237
14.3.3 Sizes in search criteria . . . . . . . . . . . . . . . . . . . . . . . . . . 238
14.3.4 Extracting emails as the result of a search . . . . . . . . . . 238
14.3.5 Moving and copying emails. . . . . . . . . . . . . . . . . . . . . . . 239
14.4 Exporting mailboxes to a target directory or synchronizingthem with the directory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
14.5 Backup and recovery. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
14.6 Repairing an index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
14.7 References to other topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
15 Performance tuning for more than 1000 simultaneous logins 245
15.1 Increasing the maximum number of clients . . . . . . . . . . . . . . . 246
Table of Contents
8
Copy
right
© O
pen
Sour
ce P
ress
15.2 Reducing login processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
15.3 Increasing the file handles. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
15.3.1 Debian/Ubuntu . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
15.3.2 openSUSE/SLES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
15.3.3 CentOS/RHEL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
15.4 Push email and IDLE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
15.5 Using the write cache in the mail storage . . . . . . . . . . . . . . . . 253
15.6 Single instance storage for attachments. . . . . . . . . . . . . . . . . . 254
16 Load-balancing cluster 257
16.1 Active/passive cluster . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
16.2 Partitioned clusters with Dovecot proxy . . . . . . . . . . . . . . . . . . 260
16.3 Active/active cluster with Dovecot Director . . . . . . . . . . . . . . . 264
16.3.1 The Dovecot Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
16.3.2 Director management during runtime. . . . . . . . . . . . . . . 269
16.4 Active/active cluster with real-time replication . . . . . . . . . . . . 271
16.4.1 Replication on the basis of doveadm via TCP/IP . . . . . . . 273
16.4.2 Replication of a public namespace . . . . . . . . . . . . . . . . . 274
16.4.3 Securing via SSL. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
16.4.4 Replication on the basis of SSH logins . . . . . . . . . . . . . . 275
16.4.5 Monitoring and maintenance with doveadm replicator . 275
16.4.6 Other things you need to consider . . . . . . . . . . . . . . . . . 276
17 Autoconfig and Autodiscover – automatic configuration ofemail clients 277
17.1 Autoconfig with Thunderbird . . . . . . . . . . . . . . . . . . . . . . . . . . . 278
17.2 Autodiscover with Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
17.3 Automatic configuration for Mac OS, iOS and WindowsLiveMail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
17.4 Dynamic configuration scripts in PHP and Python . . . . . . . . . . 284
18 Webmailer as a complementary service 285
Table of Contents
9
Copy
right
© O
pen
Sour
ce P
ress
18.1 Squirrelmail and Horde/IMP . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
18.2 Roundcube . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
18.2.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
18.2.2 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
18.3 up-imapproxy – fast access through session caching . . . . . . . . 293
19 Migrating IMAP servers 295
19.1 Different ways to migrate a mail server . . . . . . . . . . . . . . . . . . 296
19.1.1 Migrations at file level . . . . . . . . . . . . . . . . . . . . . . . . . . 297
19.1.2 Migrations using the POP3/IMAP protocol . . . . . . . . . . . 299
19.2 Migration with imapsync . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
19.3 Transparent migrations with imapc . . . . . . . . . . . . . . . . . . . . . . 302
19.4 Changes to folder names . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304
19.5 Determining plain text passwords . . . . . . . . . . . . . . . . . . . . . . . 305
20 Enterprise features and support 309
20.1 obox: Dovecot object storage . . . . . . . . . . . . . . . . . . . . . . . . . . 310
20.2 Configuration of various object storages. . . . . . . . . . . . . . . . . . 312
20.3 Future prospects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
III Appendix 319
A doveadm – the Finnish army knife 321
B IMAP command reference 327
B.1 IMAP commands available at any time . . . . . . . . . . . . . . . . . . . 328
B.2 Commands in the not-authenticated state . . . . . . . . . . . . . . . . 329
B.3 Commands in the authenticated state . . . . . . . . . . . . . . . . . . . . 330
B.4 Commands in the Selected state . . . . . . . . . . . . . . . . . . . . . . . . 336
B.5 IMAP extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 348
B.6 Experimental commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 350
Table of Contents
10
Copy
right
© O
pen
Sour
ce P
ress
C POP3 command reference 351
C.1 An overview of all commands. . . . . . . . . . . . . . . . . . . . . . . . . . . 352
C.2 Optional commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353
D The Sieve script language 355
D.1 Linking conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
D.2 Overview of the various Sieve conditions. . . . . . . . . . . . . . . . . . 357
D.3 Match types for header and address . . . . . . . . . . . . . . . . . . . . . 359
D.4 Sieve actions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 360
Index 363
Table of Contents
11
Copy
right
© O
pen
Sour
ce P
ress
9The IMAP namespace and shared
folders
Shared folders are an extremely useful way of enabling a team to collab-orate on a project. You don’t need an elaborate groupware solution orany other tricks – IMAP provides everything you need.
A user can use IMAP ACLs (access control lists) to grant another userpermission to access one of his folders. He can make detailed decisionson the permissions that the other user should be granted:
l (lookup)The folder is visible and the user can subscribe to it.
r (read)The user can select the folder for reading.
Chap
ter
169
Copy
right
© O
pen
Sour
ce P
ress
w (write)The user can save/change message flags and keywords except for\Seen and \Deleted.
s (write-seen)The \Seen flag can be set/modified.
t (write-deleted)The \Deleted flag can be set/modified.
i (insert)Messages can be written or copied to this folder.
p (post)Messages can be delivered to this folder via dovecot-lda or LMTP,for example as the result of Sieve filtering.
e (expunge)Messages can be marked for deletion in this folder.
k (create)Under this folder, the user can create new folders and rename exist-ing ones. Renaming is only possible with delete permissions.
x (delete)The user can delete this folder.
a (admin)The user has administration permissions for this folder (and is there-fore allowed to set ACLs).
The great thing is that the user can set ACLs in his email client withoutassistance from the administrator. Older email clients do not supportACLs, and Thunderbird still requires installation of the Imap-ACL-Exten-sion from menu item Extras ‣ Add-ons. In principle, however, modernemail clients can handle ACLs for reading and writing operations. Withan acl plugin, current versions of the Roundcube webmailer also sup-port the reading and setting of ACLs in the IMAP plugin.ACL settings are not always easy to find – in Thunderbird, for example,you can find them by right-clicking on the folder in question and thenchoosing Properties ‣ Sharing ‣ Privileges.Please note that the user should specify the login name of the user whois to be granted access to the directories. In setups where login name and
9 The IMAP namespace and shared folders
170
Copy
right
© O
pen
Sour
ce P
ress
email address are identical, this is simple and easy to do: you provide ac-cess to a different email address, and that’s all there is to it.If, however, short names are used as login names, the person grantingaccess must first ask the recipient for his login name in order to enterthe ACL rule correctly. I have encountered quite a few data protectionissues at universities and colleges where the login name also containedthe student’s matriculation number; of course, that is a fundamentalproblem anyway. Exam results are often posted “anonymously” on no-tice boards under students' matriculation numbers.All in all, shared folders are another reason why I think users should login with their email address. Unfortunately that is not always possible,and in some setups it does not even make sense.
9.1 Necessary preparationShared folders work in Dovecot in the following way:
1. If a user logs in and the acl plugin is active, Dovecot consults a dic-tionary to determine whether the other users have granted access tothis user.
2. The IMAP process of the user who is logged in uses a userdb queryto determine the home directories of the other users.
3. Then Dovecot looks at the dovecot-acl-list file of these users,which lists the folders for which ACL permissions have been set.
4. In the last step, Dovecot evaluates the dovecot-acl file in the respec-tive folders; this file actually contains the list of ACLs for the folderin question.
There is a simple reason for this rather complicated method. If you havea system where many tens of thousands of users each own thousands offolders, Dovecot is unable to search all folders for existing ACLs when auser logs in. It has to use central content directories to determine quicklywhich folders are even a possible option, i.e. where an ACL permissionfor the user who is logged in may be hidden.When a user revokes the permissions he has granted to another user, theoriginal entry in the central directory remains. Dovecot does not tidy upthere, even if that means it has to search the home directory of the userwho originally shared the folder. That is (slightly) inefficient but not real-ly a problem, because a hundred email directories do not really make a
9.1 Necessary preparation
171
Copy
right
© O
pen
Sour
ce P
ress
difference during login. The important thing is that this mechanism ex-cludes hundreds of thousands of other directories!First you need to prepare Dovecot for the central dictionary /var/lib/dovecot/db/shared-mailboxes.db. The (unprivileged) IMAP process ofthe logged-in user must also be able to read and write to it. As a tempo-rary file has to be set up when this file is changed, you have to modifythe file permissions of the whole /var/lib/dovecot/db folder. If, as rec-ommended in this book, you use the cental uid vmail for all users, theprocedure is simple:flash:~ # mkdir /var/lib/dovecot/dbflash:~ # chown vmail:vmail /var/lib/dovecot/db
During operation, make sure that this file is kept in sync on all clusternodes.¹You need to set up another auth socket with unprivileged access permis-sions so that the IMAP process of the logged-in user can use a userdbquery to determine other home directories. Modify the existing auth-userdb socket in 10-master.conf as follows:service auth { unix_listener auth-userdb { #mode=0660 user = vmail group = vmail }}
Now you need to make sure that the acl plugin is loaded for all modules.Add the acl plugin to mail_plugins in the 10-mail.conf file:mail_plugins = [...] acl
Then add the imap_acl plugin to the IMAP module in the 20-imap.conffile. After all, you want the user to be able to manage his access permis-sions via the IMAP protocol:protocol imap { mail_plugins = $mail_plugins [...] imap_acl}
Finally, tell the acl plugin in the 90-acl.conf file where to create thedictionary:plugin { acl = vfile
1 http://wiki2.dovecot.org/SharedMailboxes/Shared shows how to provide thedictionary on an SQL server.
9 The IMAP namespace and shared folders
172
Copy
right
© O
pen
Sour
ce P
ress
acl_shared_dict = file:/var/lib/dovecot/db/shared-mailboxes.db}
Later on there will be entries according to the following pattern:shared/shared-boxes/user/[email protected]/[email protected]
The shared-mailboxes file should be read from right to left: Ivonne hasgranted Peer access. The remaining entry, shared/shared-boxes/user/,is always the same and does not have any deeper meaning at the mo-ment.If the acl plugin is active, doveadm in version 2.2 or higher is also famili-ar with corresponding acl commands (see Section 9.6.1).
9.2 Definition of a shared namespace
Now you can set up a special IMAP namespace under which the sharedfolders belonging to other users are displayed. In addition to the name-space for the INBOX, the /etc/dovecot/10-mail.conf file contains ashared type namespace that you have to activate and modify as shownhere:namespace inbox { type = private hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = separator = . subscriptions = yes}
namespace { type = shared hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/Maildir:INDEX=%h/shared/%%u:CONTROL=%h/shared/%%u prefix = shared/%%u/ separator = . subscriptions = yes}
In Dovecot, %h refers to a user’s home directory and %u refers to his username, while double percentage signs in a shared namespace always referto the data of the other user, i.e. the one sharing the folder. %%h/Maildir
9.2 Definition of a shared namespace
173
Copy
right
© O
pen
Sour
ce P
ress
therefore refers to the Maildir directory containing the shared email da-ta.If user Peer grants user Ivonne access to his folder INBOX.Personal.Va-cation, the folder would be displayed to Ivonne in this configuration asshared +-- peer +-- INBOX +-- Personal +-- Vacation
and access to this folder would be redirected to Peer’s home directory.However, the index files and the control files (such as the subscriptionsfile) remain with user Peer, stored in a directory named shared/<other-username>.Once you have completed all these steps and reloaded Dovecot aftercompleting the changes to the configuration, your users can grant accessto their IMAP folders. But there are still a few traps to remember and afew decisions to make.
9.3 The right hierarchy separator for a sharednamespace
In practice, the configuration shown above will cause problems if theusers' login names contain a point, as in [email protected]. Asthe point is also used as a hierarchy separator, the resulting hierarchy forthe email client would beshared +-- p +-- heinlein@example +-- com +-- INBOX +-- Personal +-- Vacation
If you cannot ensure that user names will never contain a point, I strong-ly advise you to change the hierarchy separator from a point to “/”.INBOX.Personal.Vacation becomes INBOX/Personal/Vacation, and theemail client is able to display a shared-folder release shared/[email protected]/INBOX/Personal/Vacation with the right hierar-chies.²
2 There is a plugin called listescape, which can escape dots that are not supposed tobe hierarchy separators in such a way that the client shows them as a dot and does
9 The IMAP namespace and shared folders
174
Copy
right
© O
pen
Sour
ce P
ress
If you are just setting up your email system, it is easy to change the hier-archy separator. Email clients will recognize that you have done so andadapt to that fact from the start. The IMAP protocol is set up so that anIMAP server can choose its own hierarchy separator (and the client candecide which separator it will use in its final display). In our example,you would specify separator = / in both namespaces (!):namespace inbox { type = private hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = separator = / subscriptions = yes}
namespace { type = shared hidden = no ignore_on_failure = no inbox = no list = children location = maildir:%%h/Maildir:INDEX=%h/shared/%%u:CONTROL=%h/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes}
If you make this change on a system that is already running and con-tains relevant email data, make sure you observe the following points:
■ In the subscription files, where Dovecot lists the folders the userhas subscribed to, the folder names can still contain a point. Dovecotcan understand and translate that.
■ However, you need to adapt the folder names in Sieve filtering rules.fileinto INBOX.Mailinglists.Dovecot now becomes fileintoINBOX/Mailinglists/Dovecot. Don’t replace all points in Sievescripts, otherwise you may destroy email addresses and autorespond-er texts. Make sure that you only replace points in lines that also con-tain the fileinto command.
not use them as a hierarchy separator: http://wiki.dovecot.org/Plugins/Listescape. However, I still encountered some strange problems with various email cli-ents, so I do not consider this a feasible option.
9.3 The right hierarchy separator for a shared namespace
175
Copy
right
© O
pen
Sour
ce P
ress
The following sed command performs this task, but it outputs the modi-fied files only on the screen:flash:~ # sed '/fileinto/ s/\./\//g' sievescript.txt
Check the output of sed and then have the actual changes performed inthe files via parameter `-i´.flash:~ # sed -i '/fileinto/ s/\./\//g' sievescript.txt
9.4 Shared folders in mdbox or the auto:-modeBut be careful. Unlike in the Maildir format, the dovecot.index.* indexfiles are essential with mdbox so that Dovecot can identify the positionof the emails in the m files again (see Section 7.3).Unlike Maildir, the mdbox format does not allow INDEX and CONTROL to betwisted towards a directory of the share recipient in the shared name-space.So the option below would be wrong for mdbox:namespace { type = shared prefix = shared/%%u/ location = mdbox:%%h/mdbox:INDEX=%h/shared/%%u:CONTROL=%h/shared/%%u}
The right version for mdbox is:namespace { type = shared prefix = shared/%%u/ location = mdbox:%%h/mdbox}
Dovecot can then use the share giver’s dovecot.cache.* and find theemails 's mdbox directory.A similar principle applies if you allow Dovecot to determine the type ofthe target directory automatically, so if you are using the auto: mode(see Section 7.5):mail_location=auto:
There can be no parameter after auto:. You therefore cannot use auto:%%h to specify the remote home directory. Even shared namespaces re-quire nothing other than auto for the auto mode:namespace { type = shared separator = /
9 The IMAP namespace and shared folders
176
Copy
right
© O
pen
Sour
ce P
ress
prefix = shared/%%u/ location = auto: subscriptions = yes list = children}
9.5 Folders parallel to the INBOXIf you have not done so before, you should think carefully about thestructure of the inbox namespace when you introduce shared folders.The IMAP protocol itself does not specify whether IMAP folders may ex-ist in parallel to the INBOX or have to be underneath it in the hierarchy.This subject was discussed in Section 4.4.1, so you may want to read upon it again there.Considering the two points made there, I would always advise you to al-low folders only underneath the INBOX. Particularly if you are building anew system from scratch, you should set it up to be future-proof: there isno real reason not to, and you do not need to perform any migrationslater on.Set the suitable prefix in namespace inbox. Remember that the prefixmust end with the hierarchy separator. Depending on the setup, the en-try should be INBOX. or INBOX/ – and not just INBOX:namespace inbox { type = private hidden = no ignore_on_failure = no inbox = yes list = yes location = prefix = INBOX/ separator = / subscriptions = yes}
If you decide to adapt the namespace on a system that has grown overtime, make sure you observe the instructions in Chapter 19.
9.6 Public foldersWhile shared folders involve individual users granting one another ac-cess, public folders are not tied to a single user as the share giver, but areinstead set up and shared centrally by an administrator at file level.At first they are accessible to all users in the system, almost like a noticeboard, but the administrator can also use ACLs to determine which users
9.5 Folders parallel to the INBOX
177
Copy
right
© O
pen
Sour
ce P
ress
have access to which public folders, and what kind of permissions theyhave for these folders.In addition to the namespaces described earlier, inbox and shared, youcan enter the public namespace in 10-mail.conf (public is currentlynot yet prepared in the configuration files):namespace { type = public separator = / prefix = Public/ location = maildir:/srv/vmail/public subscriptions = yes}
Make sure that separator and prefix match the rest of your namespace.If you are operating Dovecot in an active/active replication, please alsoobserve Section 16.4.2.Now create the directory for your public namespace. Make sure that ev-erything is accessible to user vmail and this path does not accidentallycross that of another user.flash:/srv/vmail # mkdir publicflash:/srv/vmail # cd publicflash:/srv/vmail/public # mkdir cur new tmpflash:/srv/vmail/public # mkdir ".Newsletters"flash:/srv/vmail/public # mkdir ".Menu for canteen"flash:/srv/vmail/public # chown vmail:vmail .*flash:/srv/vmail/public # ls -latotal 16drwxr-xr-x 4 vmail vmail 4096 Aug 2 21:16 .drwxr-xr-x 4 vmail vmail 4096 Aug 2 21:15 ..drwxr-xr-x 2 vmail vmail 4096 Aug 2 21:16 .Menu for canteendrwxr-xr-x 2 vmail vmail 4096 Aug 2 21:15 .Newslettersdrwxr-xr-x 2 vmail vmail 4096 Aug 2 21:16 curdrwxr-xr-x 2 vmail vmail 4096 Aug 2 21:16 newdrwxr-xr-x 2 vmail vmail 4096 Aug 2 21:16 tmp
All users now have read, write and delete permissions for this publicfolder. That may not be quite what you want.You can use standard ACLs to manage restrictions, so your next step isto manually modify the ACL permissions in the dovecot-acl file in theMaildir. Use a special keyword, authenticated, to grant read permis-sions to all users, and then grant some privileged users write permis-sions for the public folders so they can publish messages there, for exam-ple by using drag & drop.flash:/srv/vmail/public # vi dovecot-aclauthenticated [email protected] akxeilprwts
9 The IMAP namespace and shared folders
178
Copy
right
© O
pen
Sour
ce P
ress
flash:/srv/vmail/public # chown vmail:vmail dovecot-acl
Remember to modify the ACLs for subfolders accordingly as well. Youcan simply copy the dovecot-acl file to do so.
9.6.1 Managing the public namespace with a dummyuser
In certain circumstances, you can also deliberately have the public name-space displayed in the path of a dummy user reserved for that purpose.That allows you as the administrator to integrate this special mailbox inyour email client and design the content of the public namespace. Youcan also use the email client or doveadm commands to manage the ACLsof the folder in order to control who has read access to these IMAP fold-ers and who has write access as well.Only use this trick if you have no other shared namespace. Otherwiseyour users would see your public namespace twice, once as a share bythe dummy user and once as a normal public namespace, due to the ACLshares you have set.³To use this version, set the location of the public namespace to theMaildir of the dummy user:namespace { type = public separator = / prefix = Public/ location = maildir:/srv/vmail/example.com/publicuser/Maildir subscriptions = yes}
Access the mailbox in question via your email client or webmailer anduse it to assign the required ACL permissions easily and conveniently.By default, Dovecot does not allow ACL permissions to be set for anyone(all users, even anonymous ones) or authenticated (all authenticatedusers) via the IMAP protocol. There is a great risk that users will acci-dentally grant far-reaching access.However, you can activate the keywords anyone and authenticated in90-plugin.conf in the following way if you want to manage permis-sions via your email client:
3 A crafty method would be to manually delete the necessary entry from the shareddictionary in /var/lib/dovecot/db/shared-mailboxes.db and thereby prevent theshare again.
9.6 Public folders
179
Copy
right
© O
pen
Sour
ce P
ress
plugin { acl_anyone = allow}
From Dovecot 2.2.x onwards, you can also set ACL permissions in theconsole using the Dovecot command doveadm acl:flash:~ # doveadm acl set -u [email protected] INBOX owner allflash:~ # doveadm acl set -u [email protected] INBOX authenticated ↩lookup read write-seenflash:~ # doveadm acl get -u [email protected] INBOXID Global Rightsauthenticated lookup read write-seenowner admin create delete expunge insert lookup post read ↩write write-deleted write-seen
You can then also create folders conveniently by means of the doveadmcommand:flash:~ # doveadm mailbox create -u [email protected] "INBOX/Newsletters"flash:~ # doveadm mailbox create -u [email protected] "INBOX/Menu ↩for canteen"
Hint: before you start creating folders, you should first sort out the ACLsin the public namespace. Newly created folders will then “inherit” theright ACLs, and you avoid having to repeat various tasks.
9.6.2 Display in the folder listingYou can use your own email client to check whether the public foldersare available to users, but you can also use the doveadm command toview the folders for each individual user:flash:~ # doveadm mailbox list -u [email protected]/VacationPublicPublic/Menu for canteenPublic/Newsletters
9.7 User-specific \Seen flags in shared folders andin the public namespace
In the Maildir format, the \Seen flag is coded in the file name and istherefore the same for everyone in folders used by multiple users. In themdbox format, the flag is saved in the index file rather than the file
9 The IMAP namespace and shared folders
180
Copy
right
© O
pen
Sour
ce P
ress
name, but it is still the case that all users access the same index file andso will see the same \Seen flags.From version 2.2, Dovecot allows you to save the \Seen flag in a separateindex file (in contrast to the normal process), which can be located in thehome directory of every user and separate from the actual email files.That way, the \Seen flag is saved individually for every user.You have to decide on a case-by-case basis whether individual \Seenflags are useful or inconvenient. In a team mailbox or for holiday cover,it is very useful to know what the other colleagues have already read, soindividual \Seen flags are more of a drawback. If, however, you use pub-lic or shared folders as a team distribution list to provide central infor-mation to all users, individual \Seen flags for each user are a good idea.Whether you use Maildir or mdbox, you need to add the INDEXPVT pa-rameter to the location entry in the shared or public namespace for theindividual \Seen flags, because that parameter refers to the user’s localhome directory.For a shared namespace:namespace { type = shared
# Maildir: location = maildir:%%h/Maildir:INDEX=%h/shared/%%u:CONTROL=%h/shared/%%u:INDEXPVT=%h/shared# For mdbox:# location = mdbox:%%h/mdbox:INDEXPVT=%h/mdbox/shared
[...]}
And for the public namespace:namespace { type = public
# Maildir: location = maildir:/srv/vmail/public:INDEXPVT=%h/Maildir/public
# For mdbox:# location = mdbox:%%h/mdbox:INDEXPVT=%h/mdbox/shared
[...]
}
9.7 User-specific \Seen flags in shared folders and in the public namespace
181