You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
892 lines
38 KiB
892 lines
38 KiB
/** \mainpage KMail architectural overview |
|
|
|
\section KMail design principles |
|
|
|
This file is intended to guide the reader's way through the KMail |
|
codebase. It should esp. be handy for people not hacking full-time on |
|
KMail as well as people that want to trace bugs in parts of KMail |
|
which they don't know well. |
|
|
|
Contents: |
|
- \ref kernel |
|
- \ref identity |
|
- \ref filter |
|
- \ref configuration |
|
- \ref mdn |
|
- \ref folders |
|
- \ref folderindex |
|
- \ref headers |
|
- \ref display |
|
|
|
TODO: reader, composer, messages, accounts, ... |
|
|
|
\section kernel Kernel |
|
|
|
Files: kmkernel.h, kmkernel.cpp |
|
|
|
Contact Zack Rusin <zack@kde.org> with questions... |
|
|
|
The first thing you'll notice about KMail is the extensive use of |
|
kmkernel->xxx() constructs. The "kmkernel" is a define in kmkernel.h |
|
declared as : |
|
#define kmkernel KMKernel::self() |
|
KMKernel is the central object in KMail. It's always created before |
|
any other class, therefore you are _guaranteed_ that KMKernel::self() |
|
(and therefore "kmkernel" construct) won't return 0 (null). |
|
|
|
KMKernel implements the KMailIface (our DCOP interface) and gives |
|
access to all the core KMail functionality. |
|
|
|
|
|
\section identity Identity |
|
|
|
FIXME this has moved to libkpimidentities, right? |
|
|
|
Files: identity*, kmidentity.{h,cpp}, configuredialog.cpp, |
|
signatureconfigurator.{h,cpp} |
|
|
|
Contact Marc Mutz <mutz@kde.org> on questions... |
|
|
|
Identities consists of various fields represented by |
|
QStrings. Currently, those fields are hardcoded, but feel free to |
|
implement KPIMIdentities::Identity as a map from strings to QVariants or somesuch. |
|
|
|
One part of identities are signatures. They can represent four modes |
|
(Signature::Type) of operation (disabled, text from file or command |
|
and inline text), which correspond to the combo box in the |
|
identitydialog. |
|
|
|
Identities are designed to be used through the KPIMIdentities::IdentityManager: |
|
const KPIM::Identity & ident = |
|
kmkernel->identityManager()->identityForUoidOrDefault(...) |
|
Make sure you assign to a _const_ reference, since the identityForFoo |
|
methods are overloaded with non-const methods that access a different |
|
list of identities in the manager that is used while configuring. That |
|
is known source of errors when you use identityForFoo() as a parameter |
|
to a method taking const KPIMIdentities::Identity &. |
|
|
|
WARNING: Don't hold the reference longer than the current functions |
|
scope or next return to the event loop. That's b/c the config dialog |
|
is not modal and the user may hit apply/ok anytime between calls to |
|
function that want to use the identity reference. Store the UOID |
|
instead if you need to keep track of the identity. You may also want |
|
to connect to one of the KPIMIdentities::IdentityManager::changed() or ::deleted() |
|
signals, if you want to do special processing in case the identity |
|
changes. |
|
|
|
Thus, in the ConfigureDialog, you will see non-const KPIMIdentities::Identity |
|
references being used, while everywhere else (KMMessage, |
|
IdentityCombo) const references are used. |
|
|
|
The KPIMIdentities::IdentityCombo is what you see in the composer. It's a |
|
self-updating combo box of KPIMIdentities::Identity's. Use this if you want the user |
|
to choose an identity, e.g. in the folder dialog. |
|
|
|
Ihe IdentityListView is what you see in the config dialog's identity |
|
management page. It's not meant to be used elsewhere, but is DnD |
|
enabled (well, at the time of this writing, only drag-enabled). This |
|
is going to be used to dnd identities around between KNode and KMail, |
|
e.g. |
|
|
|
The SignatureConfigurator is the third tab in the identity |
|
dialog. It's separate since it is used by the identity manager to |
|
request a new file/command if the current value somehow fails. |
|
|
|
|
|
|
|
\section filter Filter |
|
|
|
Contact Marc Mutz <mutz@kde.org> on questions... |
|
|
|
Filters consist of a search pattern and a list of actions plus a few |
|
flags to indicate when they are to be applied (kmfilter.h). |
|
They are managed in a QPtrList<KMFilter>, called KMFilterMgr. This |
|
filter magnager is responsible for loading and storing filters |
|
(read/writeConfig) and for executing them (process). The unique |
|
instance of the filter manager is held by the kernel |
|
(KMKernel::filterMgr()). |
|
|
|
The search pattern is a QPtrList of search rules (kmsearchpattern.h) and a |
|
boolean operator that defines their relation (and/or). |
|
|
|
A search rule consists of a field-QString, a "function"-enum and a |
|
"contents" or "value" QString. The first gives the header (or |
|
pseudoheader) to match against, the second says how to match (equals, |
|
consists, is less than,...) and the third holds the pattern to match |
|
against. |
|
Currently, there are two types of search rules, which are mixed |
|
together into a single class: String-valued and int-valued. The latter |
|
is a hack to enable \verbatim<size>\endverbatim and |
|
\verbatim<age in days>\endverbatim pseudo-header matching. |
|
KMSearchRules should better be organized like KMFilterActions are. |
|
|
|
A filter action (kmfilteraction.h) inherits from KMFilterAction or one |
|
of it's convenience sub-classes. They have three sub-interfaces: (1) |
|
argument handling, (2) processing and (3) parameter widget handling. |
|
Interface (1) consists of args{From,As}String(), name() and |
|
isEmpty() and is used to read and write the arguments (if any) from/to |
|
the config. |
|
Interface (2) is used by the filter manager to execute the action |
|
(process() / ReturnCode). |
|
Interface (3) is used by the filter dialog to allow editing of |
|
actions and consists of name(), label() and the |
|
*ParamWidget*(). Complex parameter widgets are collected in |
|
kmfawidget.{h,cpp}. |
|
|
|
A typical call for applying filters is |
|
|
|
\code |
|
KMKernel::filterMgr() |
|
foreach message { |
|
KMFilterMgr::process(): |
|
} |
|
\endcode |
|
|
|
|
|
\section configuration Configure Dialog |
|
|
|
Files: configuredialog*.{h,cpp} ( identitylistview.{h,cpp} ) |
|
|
|
Contact Marc Mutz <mutz@kde.org> on questions... |
|
|
|
The configuredialog is made up of pages that in turn may consist of a |
|
number of tabs. The genral rule of thumb is that each page and tab is |
|
responsible for reading and writing the config options presented on |
|
it, although in the future, that may be reduced to interacting with |
|
the corresponding config manager instead. But that won't change the |
|
basic principle. |
|
|
|
Thus, there is an abstract base class ConfigurePage (defined in |
|
configuredialog_p.h), which derives from QWidget. It has four methods |
|
of which you have to reimplement at least the first two: |
|
|
|
- void setup() |
|
Re-read the config (from the config file or the manager) and update |
|
the widgets correspondingly. That is, you should only create the |
|
widgets in the ctor, not set the options yet. The reason for that is |
|
that the config dialog, once created, is simply hidden and shown |
|
subsequently, so we need a reset-like method anyway. |
|
|
|
- void apply() |
|
Read the config from the widgets and write it into the config file |
|
or the corresponding config manager. |
|
|
|
- void installProfile() |
|
This is called when the user selected a profile and hit apply. A |
|
profile is just another KConfig object. Therefore, this method |
|
should be the same as setup(), except that you should only alter |
|
widgets for configs that really exist in the profile. |
|
|
|
For tabbed config pages, there exists a convenience class called |
|
TabbedConfigurationPage, which (as of this writing only offers the |
|
addTab() convenience method. It is planned to also provide |
|
reimplementations of setup, dismiss, apply and installProfile that just |
|
call the same functions for each tab. |
|
|
|
\section mdn MDNs |
|
|
|
Files: libkdenetwork/kmime_mdn.{h,cpp} and kmmessage.{h,cpp}, mostly |
|
|
|
Contact Marc Mutz <mutz@kde.org> on questions... |
|
|
|
MDNs (Message Disposition Notifications; RFC 2298) are a way to send |
|
back information regarding received messages back to their |
|
sender. Examples include "message read/deleted/forwarded/processed". |
|
|
|
The code in kmime_mdn.{h,cpp} is responsible for creating the |
|
message/disposition-notification body part (2nd child of |
|
multipart/report that makes the MDN) and for providing the template |
|
for human-readable text that goes into the text/plain part (1st child |
|
of the multipart/report). |
|
|
|
The code in KMMessage::createMDN() actually constructs a message |
|
containing a MDN for this message, using the kmime_mdn helper |
|
functions. It starts by checking the index for an already sent MDN, |
|
since the RFC demands that MDNs be sent only once for every |
|
message. If that test succeeds, it goes on to check various other |
|
constraints as per RFC and if all goes well the message containing the |
|
multipart/report is created. |
|
|
|
If you need to use this functionality, see KMReaderWin::touchMsg() and |
|
KMFilterAction::sendMDN() for examples. The touchMsg() code is invoked |
|
on display of a message and sends a "displayed" MDN back (if so |
|
configured), whereas the KMFilterAction method is a convenience helper |
|
for the various filter actions that can provoke a MDN (move to trash, |
|
redirect, forward, ...). |
|
|
|
|
|
\section folders Folders |
|
|
|
Files: kmfolder*.{h,cpp}, folderstorage.{h,cpp} and *job.{h,cpp} |
|
|
|
Contact Zack Rusin <zack@kde.org> with questions... |
|
|
|
The collaboration among KMail folder classes looks |
|
as follows : |
|
|
|
<pre> |
|
KMFolderNode |
|
/ \ |
|
/ \ |
|
KMFolderDir \ |
|
KMFolder |
|
. |
|
. |
|
v |
|
FolderStorage |
|
| |
|
| |
|
KMFolderIndex |
|
| |
|
| |
|
---< actual folder types: KMFolderImap, KMFolderMbox... >-- |
|
</pre> |
|
|
|
At the base KMail's folder design starts with KMFolderNode which |
|
inherits QObject. KMFolderNode is the base class encapsulating |
|
common folder properties such as the name and a boolean signifying whether |
|
the folder is a folder holding mail directly or a KMFolderDir. |
|
KMFolderNode's often do not have an on-disk representation, they are |
|
entities existing only within KMail's design. |
|
|
|
KMFolder acts as the runtime representation of a folder with the physical |
|
storage part being represented by a member of type FolderStorage. |
|
KMFolder and FolderStorage have many functions with the same names and |
|
signatures, but there is no inheritance. |
|
KMFolderIndex contains some common indexing functionality for physical folders. |
|
Subclasses of KMFolderIndex finally interact directly with physical storage |
|
or with storage providers over the network. |
|
|
|
KMFolderDir is a directory abstraction which holds KMFolderNode's. |
|
It inherits KMFolderNode and KMFolderNodeList which is a QPtrList<KMFolderNode>. |
|
A special case of a KMFolderDir is KMFolderRootDir; it represents |
|
the toplevel KMFolderDir in KMail's folder hierarchy. |
|
|
|
KMFolderDir's contents are managed by KMFolderMgr's. |
|
KMail contains four main KMFolderMgr's. They can be |
|
accessed via KMKernel ( the "kmkernel" construct ). Those methods are : |
|
1) KMFolderMgr *folderMgr() - which returns the folder manager for |
|
the folders stored locally. |
|
2) KMFolderMgr *imapFolderMgr() - which returns the folder manager |
|
for all imap folders. They're handled a little differently because |
|
for all imap messages only headers are cached locally while the |
|
main contents of all messages is kept on the server. |
|
3) KMFolderMgr *dimapFolderMgr() - which returns disconnected IMAP (dimap) |
|
folder manager. In dimap, both the headers and a copy of the full message |
|
are cached locally. |
|
4) KMFolderMgr *searchFolderMgr() - which returns the folder manager |
|
for search folders (folders created by using the "find |
|
messages" tool). Other email clients call this type of folder |
|
"virtual folders". |
|
|
|
FolderJob classes - These classes allow asynchronous operations on |
|
KMFolder's. You create a Job on the heap, connect to one of its |
|
signals and wait for the job to finish. Folders serve as FolderJob |
|
factories. For example, to retrieve the full message from a folder |
|
you do : |
|
|
|
\code |
|
FolderJob *job = folderParent->createJob( aMsg, tGetMessage ); |
|
connect( job, SIGNAL(messageRetrieved(KMMessage*)), |
|
SLOT(msgWasRetrieved(KMMessage*)) ); |
|
job->start(); |
|
\endcode |
|
|
|
|
|
\section folderindex Index (old) |
|
|
|
Files: kmfolderindex.{h,cpp} and kmmsg{base,info}.{h,cpp} |
|
|
|
Contact Marc Mutz <mutz@kde.org> or |
|
Till Adam <adam@kde.org> or |
|
Don Sanders <sanders@kde.org> |
|
with questions... |
|
|
|
index := header *entry |
|
|
|
|
|
header := magic LF NUL header-length byte-order-marker sizeof-long |
|
|
|
magic := "# KMail-Index V" 1*DIGITS |
|
|
|
header-length := quint32 |
|
|
|
byte-order-marker := quint32( 0x12345678 ) |
|
|
|
sizeof-long := quint32( 4 / 8 ) |
|
|
|
|
|
entry := tag length value |
|
|
|
tag := quint32 ; little endian (native?) |
|
|
|
length := quint16 ; little endian (native?) |
|
|
|
value := unicode-string-value / ulong-value |
|
|
|
unicode-string-value := 0*256QChar ; network-byte-order |
|
|
|
ulong-value := unsigned_long ; little endian |
|
|
|
Currently defined tag values are: |
|
|
|
<pre> |
|
Msg*Part num. val type obtained by: |
|
|
|
No 0 u - |
|
From 1 u fromStrip().trimmed() |
|
Subject 2 u subject().trimmed() |
|
To 3 u toStrip().trimmed() |
|
ReplyToIdMD5 4 u replyToIdMD5().trimmed() |
|
IdMD5 5 u msgIdMD5().trimmed() |
|
XMark 6 u xmark().trimmed() |
|
Offset 7 l folderOffset() (not only mbox!) |
|
LegacyStatus 8 l mLegacyStatus |
|
Size 9 l msgSize() |
|
Date 10 l date() |
|
File 11 u fileName() (not only maildir!) |
|
CryptoState 12 l (signatureState() << 16) | encryptionState()) |
|
MDNSent 13 l mdnSentState() |
|
ReplyToAuxIdMD5 14 u replyToAuxIdMD5() |
|
StrippedSubject 15 u strippedSubjectMD5().trimmed() |
|
Status 16 l status() |
|
</pre> |
|
|
|
u: unicode-string-value; l: ulong-value |
|
|
|
Proposed new (KDE 3.2 / KMail 1.6) entry format: |
|
|
|
index := header *entry |
|
|
|
entry := sync 1*( tag type content ) crc-tag crc-value |
|
|
|
sync := quint16 (32?) ; resync mark, some magic bit pattern |
|
; (together with preceding crc-tag provides |
|
; 24(40)bits to resync on in case of index |
|
; corruption) |
|
|
|
tag := quint8 |
|
|
|
type := quint8 |
|
|
|
content := variable-length-content / fixed-length-content |
|
|
|
crc-tag := tag type ; tag=CRC16, type=CRC16 |
|
|
|
crc-value := quint16 ; the CRC16 sum is calculated over all of |
|
; 1*( tag type content ) |
|
|
|
variable-length-content := length *512byte padding |
|
|
|
padding := *3NUL ; make the string a multiple of 4 octets in length |
|
|
|
fixed-length-content := 1*byte |
|
|
|
length := quint16 (quint8?) |
|
|
|
byte := quint8 |
|
|
|
The type field is pseudo-structured: |
|
|
|
<pre> |
|
bit: 7 6 5 4 3 2 1 0 |
|
+-----+-----+-----+-----+-----+-----+-----+-----+ |
|
| uniq | chunk | len | |
|
+-----+-----+-----+-----+-----+-----+-----+-----+ |
|
</pre> |
|
|
|
uniq: 3 bits = max. 8 different types with same chunk size: |
|
|
|
<pre> |
|
for chunk = (0)00 (LSB(base)=0: octets): |
|
00(0) Utf8String |
|
01(0) BitField |
|
10(0) reserved |
|
11(0) Extend |
|
|
|
for chunk = (1)00 (LSB(base)=1: 16-octet blocks): |
|
00(1) MD5(Hash/List) |
|
01(1) reserved |
|
10(1) reserved |
|
11(1) Extend |
|
|
|
for chunk = 01 (shorts): |
|
000 Utf16String |
|
001-110 reserved |
|
111 Extend |
|
|
|
for chunk = 10 (int32): |
|
000 Utf32String (4; not to be used yet) |
|
001 Size32 |
|
010 Offset32 |
|
011 SerNum/UOID |
|
100 DateTime |
|
101 Color (QRgb: (a,r,g,b)) |
|
110 reserved |
|
111 Extend |
|
|
|
for chunk = 11 (int64): |
|
000 reserved |
|
001 Size64 |
|
010 Offset64 |
|
011-110 reserved |
|
111 Extend |
|
</pre> |
|
|
|
len: length in chunks |
|
000 (variable width -> quint16 with the real width follows) |
|
001..111: fixed-width data of length 2^len (2^1=2..2^6=128) |
|
|
|
You find all defined values for the type field in indexentrybase.cpp |
|
|
|
Currently defined tags are: |
|
|
|
<pre> |
|
tag type content |
|
|
|
DateSent DateTime Date: |
|
DateReceived DateTime last Received:'s date-time |
|
FromDisplayName String decoded display-name of first From: addr |
|
ToDisplayName String dto. for To: |
|
CcDisplayName String dto. for Cc: |
|
FromAddrSpecs String possibly IMAA-encoded, comma-separated addr-spec's of From: |
|
ToAddrSpecs String dto. for To: |
|
CcAddrSpecs String dto. for Cc: |
|
Subject String decoded Subject: |
|
BaseSubjectMD5 String md5(utf8(stripOffPrefixes(subject()))) |
|
BodyPeek String body preview |
|
MaildirFile String Filename of maildir file for this messagea |
|
MBoxOffset Offset(64) Offset in mbox file (pointing to From_) |
|
MBoxLength Size(64) Length of message in mbox file (incl. From_) |
|
Size Size(64) rfc2822-size of message (in mbox: excl. From_) |
|
Status BitField (see below) |
|
MessageIdMD5 MD5Hash MD5Hash of _normalized_ Message-Id: |
|
MDNLink SerialNumber SerNum of MDN received for this message |
|
DNSLink SerialNumber SerNUm of DSN received for this message |
|
ThreadHeads SerialNumberList MD5Hash's of all (so far discovered) |
|
_top-level thread parents_ |
|
ThreadParents SerialNumberList MD5Hash's of all (so far discovered) |
|
thread parents |
|
</pre> |
|
|
|
|
|
"String" is either Utf8String or (Utf16String or Latin1String), |
|
depending on content |
|
|
|
Currently allocated bits for the Status BitField are: |
|
|
|
<pre> |
|
Bit Value: on(/off) (\\imapflag) |
|
|
|
# "std stati": |
|
0 New (\\Recent) |
|
1 Read (\\Seen) |
|
2 Answered (\\Answered) |
|
3 Deleted (\\Deleted) |
|
4 Flagged (\\Flagged) |
|
5 Draft (\\Draft) |
|
6 Forwarded |
|
7 reserved |
|
|
|
# message properties: |
|
8 HasAttachments |
|
9 MDNSent ($MDNSent) |
|
10..11 00: unspecified, 01: Low, 10: Normal, 11: High Priority |
|
12..15 0001: Queued, 0010: Sent, 0011: DeliveryDelayed, |
|
0100: Delivered, 0101: DeliveryFailed, |
|
0110: DisplayedToReceiver, 0111: DeletedByReceiver, |
|
1001: ProcessedByReceiver, 1010: ForwardedByReceiver, |
|
1011-1110: reserved |
|
1111: AnswerReceived |
|
|
|
# signature / encryption state: |
|
16..19 0001: FullyEncrypted, 0010: PartiallyEncrypted, 1xxx: Problem |
|
20..23 0001: FullySigned, 0010: PartiallySigned, 1xxx: Problem |
|
|
|
# "workflow stati": |
|
24..25 01: Important, 10: ToDo, 11: Later (from Moz/Evo) |
|
26..27 01: Work, 10: Personal, 11: reserved (dto.) |
|
28..29 01: ThreadWatched, 10: ThreadIgnored, 11: reserved |
|
30..31 reserved |
|
</pre> |
|
|
|
All bits and combinations marked as reserved MUST NOT be altered if |
|
set and MUST be set to zero (0) when creating the bitfield. |
|
|
|
|
|
\section headers Headers (Threading and Sorting) |
|
|
|
Contact Till Adam <adam@kde.org> or |
|
Don Sanders <sanders@kde.org> |
|
with questions... |
|
|
|
Threading and sorting is implemented in kmheaders.[cpp|h] and headeritem.[cpp|h] |
|
and involves a handfull of players, namely: |
|
|
|
class KMHeaders: |
|
this is the listview that contains the subject, date etc of each mail. |
|
It's a singleton, which means there is only one, per mainwidget headers |
|
list. It is actually a member of KMMainwidget and accessible there. |
|
|
|
class KMail::HeaderItem: |
|
these are the [Q|K]ListViewItem descendend items the KMHeaders listview |
|
consists of. There's one for each message. |
|
|
|
class KMail::SortCacheItem: |
|
these are what the threading and sorting as well as the caching of those |
|
operate on. Each is paired with a HeaderItem, such that each holds a |
|
pointer to one, and vice versa, each header item holds a pointer to it's |
|
associated sort cache item. |
|
|
|
.sorted file: |
|
The order of the sorted and threaded (if threading is turned on for this |
|
folder) messages is cached on disk in a file named .$FOLDER.index.sorted |
|
so if, for example the name of a folder is foo, the associated sorting |
|
cache file would be called ".foo.index.sorted". |
|
For each message, its serial number, that of its parent, the length of its |
|
sorting key, and the key itself are written to this file. At the start of |
|
the file several per folder counts and flags are cached additionally, |
|
immediately after a short file headers. The entries from the start of the |
|
file are: |
|
- "## KMail Sort V%04d\n\t" (magic header and version string) |
|
- byteOrder flag containing 0x12345678 for byte order testing |
|
- column, the sort column used for the folder |
|
- ascending, the sort direction |
|
- threaded, is the view threaded or is it not? |
|
- appended, have there been items appended to the file (obsolete?) |
|
- discovered_count, number of new mail discovered since the last sort file |
|
generation |
|
- sorted_count, number of sorted messages in the header list |
|
|
|
What is used for figuring out threading? |
|
- messages can have an In-Reply-To header that contains the message id of |
|
another message. This is called a perfect parent. |
|
- additionally there is the References header which, if present, holds a |
|
list of message ids that the current message is a follow up to. We |
|
currently use the second to last entry in that list only. See further |
|
down for the reasoning behind that. |
|
- If the above two fail and the message is prefixed (Re: foo, Fwd: foo etc.) |
|
an attempt is made to find a parent by looking for messages with the same |
|
subject. How that is done is explained below as well. |
|
|
|
For all these comparisons of header contents, the md5 hashes of the headers |
|
are used for speed reasons. They are stored in the index entry for each |
|
message. All data structures described below use md5 hash strings unless |
|
stated otherwise. |
|
|
|
Strategy: |
|
When a folder is opened, updateMessageList is called, which in turn calls |
|
readSortOrder where all the fun happens. If there is a .sorted file at the |
|
expected location, it is openend and parsed. The header flags are read in |
|
order to figure out the state in which this .sorted file was written. This |
|
means the sort direction, whether the folder is threaded or not etc. |
|
FIXME: is there currently sanity checking going on? |
|
Now the file is parsed and for each message in the file a SortCacheItem is |
|
created, which holds the offset in the .sorted file for this message as well |
|
as it's sort key as read from the file. That sort cache item is entered into |
|
the global mSortCache structure (member of the KMHeaders instance), which is |
|
a QMemArray<SortCacheItem *> of the size mFolder->count(). Note that this |
|
is the size reported by the folder, not as read from the .sorted file. The |
|
mSortCache (along with some other structures) is updated when messages are |
|
added or removed. More on that further down. |
|
As soon as the serial number is read from the file, that number is looked up |
|
in the message dict, to ensure it is still in the current folder. If not, it |
|
has been moved away in the meantime (possibly by outside forces such as |
|
other clients) and a deleted counter is incremented and all further |
|
processing stopped for this message. |
|
The messages parent serial number, as read from the sorted file is then |
|
used to look up the parent and reset it to -1 should it not be in the |
|
current folder anymore. -1 and -2 are special values that can show up |
|
as parent serial numbers and are used to encode the following: |
|
-1 means this message has no perfect parent, a parent for it needs to |
|
be found from among the other messages, if there is a suitable one |
|
-2 means this message is top level and will forever stay so, no need |
|
to even try to find a parent. This is also used for the non-threaded |
|
case. These are messages that have neither an In-Reply-To header nor |
|
a References header and have a subject that is not prefixed. |
|
In case there is a perfect parent, the current sort cache item is |
|
appended to the parents list of unsorted children, or to that of |
|
root, if there is not. A sort cache item is created in the mSortCache |
|
for the parent, if it is not already there. Messages with a parent of |
|
-1 are appended to the "unparented" list, which is later traversed and |
|
its elements threaded. Messages with -2 as the parent are children of |
|
root as well, as noted above, and will remain so. |
|
|
|
Once the end of the file is reached, we should have a nicely filled |
|
mSortCache, containing a sort cache item for each message that was in the |
|
sorted file. Messages with perfect parents know about them, top level |
|
messages know about that as well, all others are on a list and will be |
|
threaded later. |
|
|
|
Now, what happens when messages have been added to the store since the last |
|
update of the .sorted file? Right, they don't have a sort cache item yet, |
|
and would be overlooked. Consequently all message ids in the folder from 0 |
|
to mFolder->count() are looked at and a SortCacheItem is created for the |
|
ones that do not have one yet. This is where all sort cache items are created |
|
if there was no sorted file. The items created here are by definition un- |
|
sorted as well as unparented. On creation their sort key is figured out as |
|
well. |
|
|
|
The next step is finding parents for those messages that are either new, or |
|
had a parent of -1 in the .sorted file. To that end, a dict of all sort |
|
cache items indexed by the md5 hash of their messsage id headers is created, |
|
that will be used for looking up sort cache items by message id. The list of |
|
yet unparented messages is then traversed and findParent() called for each |
|
element wihch checks In-Reply-To and References headers and looks up the |
|
sort cache item of those parents in the above mentioned dict. Should none be |
|
found, the item is added to a second list the items of which will be subject |
|
threaded. |
|
|
|
How does findParent() work, well, it tries to find the message pointed to by |
|
the In-Reply-To header and if that fails looks for the one pointed to by the |
|
second to last entry of the References header list. Why the second to last? |
|
Imagine the following situation in Bob's kmail: |
|
- Bob get's mail from Alice |
|
- Bob replies to Alice's mail, but his mail is stored in the Outbox, not the |
|
Inbox. |
|
- Alice replies again, so Bob now has two mails from Alice which are part of |
|
the same thread. The mail in the middle is somewhere else. We want that to |
|
thread as follows: |
|
<pre> |
|
Bob <- In-Reply-To: Alice1 |
|
============ |
|
Alice1 |
|
|_Alice <- In-Reply-To: Bob (not here), References: Alice, Bob |
|
</pre> |
|
|
|
- since the above is a common special case, it is worth trying. I think. ;) |
|
|
|
If the parent found is perfect (In-Reply-To), the sort cache items is marked |
|
as such. mIsImperfectlyThreaded is used for that purposer, we will soon see |
|
why that flag is needed. |
|
|
|
On this first pass, no subject threading is attempted yet. Once it is done, |
|
the messages that are now top-level, the current thread heads, so to speak, |
|
are collected into a second dict ( QDict< QPtrList< SortCacheItem > > ) |
|
that contains for each different subject an entry holding a list of (so far |
|
top level) messages with that subject, that are potential parents for |
|
threading by subjects. These lists are sorted by date, so the parent closest |
|
by date can be chosen. Sorting of these lists happens via insertion sort |
|
while they are built because not only are they expected to be short (apart |
|
from hard corner cases such as cvs commit lists, for which subject threading |
|
makes little sense anyhow and where it should be turned off), but also since |
|
the messages should be roughly pre sorted by date in the store already. |
|
Some cursory benchmarking supports that assumption. |
|
If now a parent is needed for a message with a certain subject, the list of |
|
top level messages with that subject is traversed and the first one that is |
|
older than our message is chosen as it's parent. Parents more than six weeks |
|
older than the message are not accepted. The reasoning being that if a new |
|
message with the same subject turns up after such a long time, the chances |
|
that it is still part of the same thread are slim. The value of six weeks |
|
is chosen as a result of a poll conducted on #kde-devel, so it's probably |
|
bogus. :) All of this happens in the aptly named: findParentBySubject(). |
|
|
|
Everthing that still has no parent after this ends up at toplevel, no further |
|
attemp is made at finding one. If you are reading this because you want to |
|
implement threading by body search or somesuch, please go away, I don't like |
|
you. :) |
|
|
|
Ok, so far we have only operated on sort cache items, nothing ui wise has |
|
happened yet. Now that we have established the parent/child relations of all |
|
messages, it's time to create HeaderItems for them for use in the header |
|
list. But wait, you say, what about sorting? Wouldn't it make sense to do |
|
that first? Exactly, you're a clever bugger, ey? Here, have a cookie. ;) |
|
Both creation of header items and sorting of the as of yet unsorted sort |
|
cache items happen at the same time. |
|
|
|
As previously mentioned (or not) each sort cache item holds a list of its |
|
sorted and one of its unsorted children. Starting with the root node the |
|
unsorted list is first qsorted, and then merged with the list of already |
|
sorted children. To achieve that, the heads of both lists are compared and |
|
the one with the "better" key is added to the list view next by creating a |
|
KMHeaderListItem for it. That header item receives both its sort key as well |
|
as its id from the sort cache item. Should the current sort cache item have |
|
children, it is added to the end of a queue of nodes to repeat these steps |
|
on after the current level is sorted. This way, a breadth first merge sort |
|
is performed on the sort cache items and header items are created at each |
|
node. |
|
|
|
What follows is another iteration over all message ids in the folder, to |
|
make sure that there are HeaderItems for each now. Should that not be the |
|
case, top level emergency items are created. This loop is also used to add |
|
all header items that are imperfectly threaded to a list so they can be |
|
reevalutated when a new message arrives. Here the reverse mapping from |
|
header items to sort cache items are made as well. Those are also necessary |
|
so the sort cache item based data structures can be updated when a message |
|
is removed. |
|
|
|
The rest of readSortOrder should make sense on itself, I guess, if not, drop |
|
me an email. |
|
|
|
What happens when a message arrives in the folder? |
|
Among other things, the msgAdded slot is called, which creates the necessary |
|
sort cache item and header item for the new message and makes sure the data |
|
structures described above are updated accordingly. If threading is enabled, |
|
the new message is threaded using the same findParent and findParentBySubject |
|
methods used on folder open. If the message ends up in a watched or ignored |
|
thread, those status bits are inherited from the parent. The message is also |
|
added to the dict of header items, the index of messages by message id and, |
|
if applicable and if the message is threaded at top level, to the list of |
|
potential parents for subject threading. |
|
|
|
After those house keeping tasks are performed, the list of as of yet imper- |
|
fectly threaded messages is traversed and our newly arrived message is |
|
considered as a new parent for each item on it. This is especially important |
|
to ensure that parents arriving out of order after their children still end |
|
up as parents. If necessary, the entries in the .sorted file of rethreaded |
|
messages are updated. An entry for the new message itself is appended to the |
|
.sorted file as well. |
|
|
|
Note that as an optimization newly arriving mail as part of a mailcheck in |
|
an imap folder is not added via msgAdded, but rather via complete reload of |
|
the folder via readSortOrder(). That's because only the headers are gotten |
|
for each mail on imap and that is so fast, that adding them individually to |
|
the list view is very slow by comparison. Thus we need to make sure that |
|
writeSortOrder() is called whenever something related to sorting changes, |
|
otherwise we read stale info from the .sorted file. The reload is triggered |
|
by the folderComplete() signal of imap folders. |
|
|
|
What happens when a message is removed from the folder? |
|
In this case the msgRemoved slot kicks in and updates the headers list. First |
|
the sort cache item and header item representing our message are removed from |
|
the data structures and the ids of all items after it in the store decre- |
|
mented. Then a list of children of the message is assembled containing those |
|
children that have to be reparented now that our message has gone away. If |
|
one of those children has been marked as toBeDeleted, it is simply added to |
|
root at top level, because there is no need to find a parent for it if it is |
|
to go away as well. This is an optimization to avoid rethreading all |
|
messages in a thread when deleting several messages in a thread, or even the |
|
whole thread. The KMMoveCommand marks all messages that will be moved out of |
|
the folder as such so that can be detected here and the useless reparenting |
|
can be avoided. Note that that does not work when moving messages via filter |
|
action. |
|
|
|
That list of children is then traversed and a new parent found for each one |
|
using, again, findParent and findParentBySubject. When a message becomes |
|
imperfectly threaded in the process, it is added to the corresponding list. |
|
|
|
The message itself is removed from the list of imperfectly threaded messages |
|
and its header item is finally deleted. The HeaderItem destructor destroys |
|
the SortCacheItem as well, which is hopefully no longer referenced anywhere |
|
at this point. |
|
|
|
|
|
|
|
\section display Display (reader window - new) |
|
|
|
Contact Marc Mutz <mutz@kde.org> with questions... |
|
|
|
What happens when a message is to displayed in the reader window? |
|
First, since KMMessagePart and KMMessage don't work with nested body |
|
parts, a hierarchical tree of MIME body parts is constructed with |
|
partNode's (being wrappers around KMMessagePart and |
|
DwBodyPart). This is done in KMReaderWin::parseMsg(KMMessage*). |
|
After some legacy handling, an ObjectTreeParser is instantiated and |
|
it's parseObjectTree() method called on the root |
|
partNode. ObjectTreeParser is the result of an ongoing refactoring |
|
to enhance the design of the message display code. It's an |
|
implementation of the Method Object pattern, used to break down a |
|
huuuge method into many smaller ones without the need to pass around |
|
a whole lot of paramters (those are made members |
|
instead). parseObjectTree() recurses into the partNode tree and |
|
generates an HTML representation of each node in the tree (although |
|
some can be hidden by setting them to processed beforehand or - the |
|
new way - by using AttachmentStrategy::Hidden). The HTML generation |
|
is delegated to BodyPartFormatters, while the HTML is written to |
|
HTMLWriters, which abstract away HTML sinks. One of those is |
|
KHTMLPartHTMLWriter, which writes to the KHTMLPart in the |
|
readerwindow. This is the current state of the code. The goal of the |
|
ongoing refactoring is to make the HTML generation blissfully |
|
ignorant of the readerwindow and to allow display plugins that can |
|
operate against stable interfaces even though the internal KMail |
|
classes change dramatically. |
|
|
|
To this end, we designed a new set of interfaces that allows plugins |
|
to be written that need or want to asynchronously update their HTML |
|
representation. This set of interfaces consists of the following: |
|
|
|
- @em BodyPartFormatterPlugin |
|
the plugin base class. Allows the BodyPartFormatterFactory to |
|
query it for an arbitray number of BodyPartFormatters and |
|
their associated metadata and url handlers. |
|
|
|
- @em BodyPartFormatter |
|
the formatter interface. Contains a single method format() |
|
that takes a BodyPart to process and a HTMLWriter to write the |
|
generated HTML to and returns a result code with which it can |
|
request more information or delegate the formatting back to |
|
some default processor. BodyPartFormatters are meant to be |
|
Flyweights, implying that the format() method is not allowed |
|
to maintain state between calls, but see Mememto below. |
|
|
|
- @em BodyPart |
|
body part interface. Contains methods to retrieve the content |
|
of a message part and some of the more interesting header |
|
information. This interface decouples the bodypart formatters |
|
from the implementation used in KMail (or KNode, for that |
|
matter). Also contains methods to set and retrieve a Memento, |
|
which is used by BodyPartFormatters to maintain state between |
|
calls of format(). |
|
|
|
- @em BodyPartMemento |
|
interface for opaque state storage and event handling. |
|
Contains only a virtual destructor to enable the backend |
|
processing code to destroy the object without the help of the |
|
bodypart formatter. |
|
|
|
|
|
During the design phase we identified a need for BodyPartFormatters to |
|
request their being called on some form of events, e.g. a dcop |
|
signal. Thus, the Memento interface also includes the IObserver and |
|
ISubject interfaces. If a BodyPartFormatter needs to react to a signal |
|
(Qt or DCOP), it implements the Memento interface using a QObject, |
|
connects the signal to a slot on the Memento and (as an ISubject) |
|
notifies it's IObservers when the slot is called. If a Memento is |
|
created, the reader window registers itself as an observer of the |
|
Memento and will magically invoke the corresponding BodyPartFormatter, |
|
passing along the Memento to be retrieved from the BodyPart interface. |
|
|
|
An example should make this clearer. Suppose, we want to update our |
|
display after 10 seconds. Initially, we just write out an icon, and |
|
after 10 seconds, we want to replace the icon by a "Hello world!" |
|
line. The following code does exactly this: |
|
|
|
@code |
|
class DelayedHelloWorldBodyPartFormatter |
|
: public KMail::BodyPartFormatter { |
|
public: |
|
Result format( KMail::BodyPart * bodyPart, |
|
KMail::HTMLWriter * htmlWriter ) { |
|
if ( !bodyPart->memento() ) { |
|
bodyPart->registerMemento( new DelayedHelloWorldBodyPartMemento() ); |
|
return AsIcon; |
|
} else { |
|
htmlWriter->write( "Hello, world!" ); |
|
return Ok; |
|
} |
|
} |
|
}; |
|
|
|
class DelayedHelloWorldBodyPartMemento |
|
: public QObject, public KMail::BodyPartMemento { |
|
public: |
|
DelayedHelloWorldBodyPartMemento() |
|
: QObject( 0, "DelayedHelloWorldBodyPartMemento" ), |
|
KMail::BodyPartMemento() |
|
{ |
|
QTimer::singleShot( 10*1000, this, SLOT(slotTimeout()) ); |
|
} |
|
|
|
private slots: |
|
void slotTimeout() { notify(): } |
|
|
|
private: |
|
// need to reimplement this abstract method... |
|
bool update() { return true; } |
|
}; |
|
@endcode |
|
|
|
*/ |
|
|
|
// DOXYGEN_REFERENCES = kdecore kdeui kio kparts kutils kde3support sonnet kresources kabc gpgme++ qgpgme kleo libkdepim |
|
// DOXYGEN_SET_IGNORE_PREFIX = KM K
|
|
|