incubator-ponymail.git
8 weeks agoEnforce UTF-8 character sets master
Daniel Gruno [Mon, 15 Oct 2018 09:42:54 +0000 (11:42 +0200)] 
Enforce UTF-8 character sets

This enforces UTF-8 in character sets for JSON/ATOM payloads,
which is needed for some scrapers. Fixes #479.

2 months agoBug: no need to sort after scroll
Sebb [Tue, 9 Oct 2018 20:10:47 +0000 (21:10 +0100)] 
Bug: no need to sort after scroll

This fixes #477
(reapply)

2 months agoBug: elastic.lua#scroll forces sort to use _doc
Sebb [Wed, 10 Oct 2018 23:18:26 +0000 (00:18 +0100)] 
Bug: elastic.lua#scroll forces sort to use _doc

This fixes #478

2 months agoRevert "Bug: no need to sort after scroll"
Sebb [Wed, 10 Oct 2018 09:34:20 +0000 (10:34 +0100)] 
Revert "Bug: no need to sort after scroll"

This reverts commit 557920c07620c786fa18d026e58396325f40c965.

2 months agoOops, left off sort
Sebb [Wed, 10 Oct 2018 08:44:34 +0000 (09:44 +0100)] 
Oops, left off sort

2 months agoBug: cannot download more than 10K mails to a mbox file
Sebb [Tue, 9 Oct 2018 23:20:42 +0000 (00:20 +0100)] 
Bug: cannot download more than 10K mails to a mbox file

This fixes #475

2 months agoBug: no need to sort after scroll
Sebb [Tue, 9 Oct 2018 20:10:47 +0000 (21:10 +0100)] 
Bug: no need to sort after scroll

This fixes #477

2 months agoEnh: Ensure non-printable chars are not lost ...
Sebb [Tue, 9 Oct 2018 17:45:10 +0000 (18:45 +0100)] 
Enh: Ensure non-printable chars are not lost ...

... in source and mbox output (#476)

4 months agoEnh: display buttons even if no mails are found in a month
Sebb [Mon, 16 Jul 2018 18:47:22 +0000 (19:47 +0100)] 
Enh: display buttons even if no mails are found in a month

This fixes #470

4 months agoBug: Javascript URLs must always use URL_BASE
Sebb [Mon, 16 Jul 2018 17:59:25 +0000 (18:59 +0100)] 
Bug: Javascript URLs must always use URL_BASE

THis fixes #469

4 months agoNot used
Sebb [Mon, 16 Jul 2018 13:41:06 +0000 (14:41 +0100)] 
Not used

5 months agoTypo
Sebb [Sat, 30 Jun 2018 18:29:26 +0000 (19:29 +0100)] 
Typo

6 months agoTrim trailing spaces
Sebb [Thu, 31 May 2018 22:15:56 +0000 (23:15 +0100)] 
Trim trailing spaces

6 months agoBug: setup.py uses ES library version
Sebb [Sun, 27 May 2018 11:37:29 +0000 (12:37 +0100)] 
Bug: setup.py uses ES library version

to decide what features the database supports
This fixes #464

6 months agoSimplify args processing
Sebb [Sun, 27 May 2018 01:04:30 +0000 (02:04 +0100)] 
Simplify args processing

Don't use nargs=1

6 months agoMid is not yet implemented
Sebb [Sun, 27 May 2018 00:28:37 +0000 (01:28 +0100)] 
Mid is not yet implemented

6 months agoUnnecessary check - always true
Sebb [Sun, 27 May 2018 00:21:02 +0000 (01:21 +0100)] 
Unnecessary check - always true

6 months agoSimplify args processing
Sebb [Sun, 27 May 2018 00:18:55 +0000 (01:18 +0100)] 
Simplify args processing

Don't use nargs=1
Assign values directly

6 months agoSimplify args processing
Sebb [Sun, 27 May 2018 00:07:26 +0000 (01:07 +0100)] 
Simplify args processing

Don't use nargs=1
Assign values directly

6 months agoSimplify args processing
Sebb [Sun, 27 May 2018 00:04:10 +0000 (01:04 +0100)] 
Simplify args processing

Don't use nargs=1
Assign values directly

6 months agoSimplify args processing
Sebb [Sat, 26 May 2018 23:52:18 +0000 (00:52 +0100)] 
Simplify args processing

Don't use nargs=1
Assign values directly
Use required=True for validation

6 months agoThese don't need to be global as they are read-only
Sebb [Sat, 26 May 2018 19:32:51 +0000 (20:32 +0100)] 
These don't need to be global as they are read-only

6 months agoDon't hide built-in ConnectionError
Sebb [Sat, 26 May 2018 12:32:33 +0000 (13:32 +0100)] 
Don't hide built-in ConnectionError

6 months agoSimplify by using elastic module
Sebb [Sat, 26 May 2018 12:20:30 +0000 (13:20 +0100)] 
Simplify by using elastic module

This relates to #456

6 months agoAllow use of indices.exists() method
Sebb [Sat, 26 May 2018 12:12:03 +0000 (13:12 +0100)] 
Allow use of indices.exists() method

6 months agoUse specific error
Sebb [Sat, 26 May 2018 12:10:14 +0000 (13:10 +0100)] 
Use specific error

6 months agoDon't fail engineVersion if cannot connect
Sebb [Sat, 26 May 2018 12:06:20 +0000 (13:06 +0100)] 
Don't fail engineVersion if cannot connect

6 months agoOther changes done recently
Sebb [Sat, 26 May 2018 11:42:59 +0000 (12:42 +0100)] 
Other changes done recently

6 months agoDocco
Sebb [Sat, 26 May 2018 11:29:25 +0000 (12:29 +0100)] 
Docco

6 months agoBulkThread is not a Thread
Sebb [Fri, 25 May 2018 23:08:57 +0000 (00:08 +0100)] 
BulkThread is not a Thread

Simplify - turn it into a method

6 months agoRemove temp debug
Sebb [Fri, 25 May 2018 22:34:36 +0000 (23:34 +0100)] 
Remove temp debug

6 months agoBetter name: y => goodies
Sebb [Fri, 25 May 2018 22:27:17 +0000 (23:27 +0100)] 
Better name: y => goodies

6 months agoLet the archiver import the module if it needs it
Sebb [Fri, 25 May 2018 18:12:20 +0000 (19:12 +0100)] 
Let the archiver import the module if it needs it

6 months agoPylint does not ignore names with embedded _
Sebb [Fri, 25 May 2018 18:06:05 +0000 (19:06 +0100)] 
Pylint does not ignore names with embedded _

6 months agoDon't redefine built-in 'id'
Sebb [Fri, 25 May 2018 18:05:13 +0000 (19:05 +0100)] 
Don't redefine built-in 'id'

6 months agoNarrower Exception
Sebb [Fri, 25 May 2018 17:47:16 +0000 (18:47 +0100)] 
Narrower Exception

6 months agoSimplify by using init parameters
Sebb [Fri, 25 May 2018 17:39:57 +0000 (18:39 +0100)] 
Simplify by using init parameters

6 months agoAlso need to change import-mbox for new returns
Sebb [Fri, 25 May 2018 17:26:21 +0000 (18:26 +0100)] 
Also need to change import-mbox for new returns

6 months agoDon't use instance data to pass back info from function call
Sebb [Fri, 25 May 2018 17:13:12 +0000 (18:13 +0100)] 
Don't use instance data to pass back info from function call

6 months agoFlag unused parameters with underscore prefix
Sebb [Fri, 25 May 2018 16:55:56 +0000 (17:55 +0100)] 
Flag unused parameters with underscore prefix

Also stops pylint complaining

6 months agoDocument behaviour and issue
Sebb [Fri, 25 May 2018 16:38:57 +0000 (17:38 +0100)] 
Document behaviour and issue

6 months agoFlag unused parameters with underscore prefix
Sebb [Fri, 25 May 2018 16:02:02 +0000 (17:02 +0100)] 
Flag unused parameters with underscore prefix

Also stops pylint complaining

6 months agoNote re full id format
Sebb [Fri, 25 May 2018 15:51:49 +0000 (16:51 +0100)] 
Note re full id format

6 months agoBug: archiver.py can never detect content-type: flowed
Sebb [Fri, 25 May 2018 09:55:49 +0000 (10:55 +0100)] 
Bug: archiver.py can never detect content-type: flowed

This fixes #461

6 months agoUse different name to avoid clash
Sebb [Fri, 25 May 2018 08:55:17 +0000 (09:55 +0100)] 
Use different name to avoid clash

6 months agoBug: import-mbox.py: imap code should not reset ES instance
Sebb [Thu, 24 May 2018 16:33:49 +0000 (17:33 +0100)] 
Bug: import-mbox.py: imap code should not reset ES instance

This fixes #460

6 months agopylint: logging.warn is deprecated => warning
Sebb [Thu, 24 May 2018 15:59:03 +0000 (16:59 +0100)] 
pylint: logging.warn is deprecated => warning

6 months agopylint: lazily evaluate logging parameters
Sebb [Thu, 24 May 2018 15:51:20 +0000 (16:51 +0100)] 
pylint: lazily evaluate logging parameters

6 months agopylint: Don't redefine built-in 'list'
Sebb [Thu, 24 May 2018 09:27:49 +0000 (10:27 +0100)] 
pylint: Don't redefine built-in 'list'

6 months agoBug: tmpname used before it has been set up in import-mbox
Sebb [Thu, 24 May 2018 09:13:37 +0000 (10:13 +0100)] 
Bug: tmpname used before it has been set up in import-mbox

This fixes #458

6 months agoBug: variable 'mid' used before assignment in archiver.py
Sebb [Thu, 24 May 2018 01:25:47 +0000 (02:25 +0100)] 
Bug: variable 'mid' used before assignment in archiver.py

This fixes #459

6 months agoUse common config parser module
Sebb [Wed, 23 May 2018 19:37:32 +0000 (20:37 +0100)] 
Use common config parser module

6 months agoEnh: remove duplicated code in tools scripts
Sebb [Wed, 23 May 2018 19:22:57 +0000 (20:22 +0100)] 
Enh: remove duplicated code in tools scripts

 by using elastic.py module

 This fixes #456

6 months agoAdd get method
Sebb [Wed, 23 May 2018 18:06:27 +0000 (19:06 +0100)] 
Add get method

6 months agoExit if no valid change specified
Sebb [Wed, 23 May 2018 17:46:38 +0000 (18:46 +0100)] 
Exit if no valid change specified

ALso improve display text

6 months agoDocument update
Sebb [Wed, 23 May 2018 17:44:24 +0000 (18:44 +0100)] 
Document update

6 months agoGive access to dbname loaded from config
Sebb [Wed, 23 May 2018 17:36:33 +0000 (18:36 +0100)] 
Give access to dbname loaded from config

6 months agopylint: don't redefine built-ins
Sebb [Mon, 21 May 2018 01:08:08 +0000 (02:08 +0100)] 
pylint: don't redefine built-ins

6 months agopylint: unused/duplicate imports
Sebb [Mon, 21 May 2018 01:00:35 +0000 (02:00 +0100)] 
pylint: unused/duplicate imports

6 months agopylint: unused variables
Sebb [Mon, 21 May 2018 00:54:43 +0000 (01:54 +0100)] 
pylint: unused variables

6 months agopylint: narrow catch
Sebb [Mon, 21 May 2018 00:27:52 +0000 (01:27 +0100)] 
pylint: narrow catch

6 months agopylint: narrow catch
Sebb [Mon, 21 May 2018 00:13:48 +0000 (01:13 +0100)] 
pylint: narrow catch

6 months agopylint: move doc string inside method
Sebb [Sun, 20 May 2018 15:01:27 +0000 (16:01 +0100)] 
pylint: move doc string inside method

6 months agopylint: unreachable code
Sebb [Sun, 20 May 2018 14:09:52 +0000 (15:09 +0100)] 
pylint: unreachable code

6 months agopylint: unused
Sebb [Sun, 20 May 2018 14:06:10 +0000 (15:06 +0100)] 
pylint: unused

6 months agopylint: input is a built-in
Sebb [Sun, 20 May 2018 13:51:22 +0000 (14:51 +0100)] 
pylint: input is a built-in

6 months agopylint: narrow catch
Sebb [Sun, 20 May 2018 13:49:35 +0000 (14:49 +0100)] 
pylint: narrow catch

6 months agopylint: bytes is a built-in
Sebb [Sun, 20 May 2018 13:39:34 +0000 (14:39 +0100)] 
pylint: bytes is a built-in

6 months agopylint: id is a built-in; rename
Sebb [Sun, 20 May 2018 13:30:52 +0000 (14:30 +0100)] 
pylint: id is a built-in; rename

6 months agopylint - use Python3 super() call
Sebb [Sun, 20 May 2018 13:28:39 +0000 (14:28 +0100)] 
pylint - use Python3 super() call

6 months agopylint: id is a built-in; unused import etc
Sebb [Sun, 20 May 2018 13:24:19 +0000 (14:24 +0100)] 
pylint: id is a built-in; unused import etc

6 months agoUse new config module
Sebb [Sat, 19 May 2018 12:54:04 +0000 (13:54 +0100)] 
Use new config module

Changes behaviour to expect ponymail.cfg in tools.
Only affects the new scripts missing.py and nullfav.py

6 months agoNew module to read config file
Sebb [Sat, 19 May 2018 12:52:40 +0000 (13:52 +0100)] 
New module to read config file

Assumes ponymail.cfg is in the same (tools/) directory rather
than in the current directory.

6 months agoOops, also need import
Sebb [Sat, 19 May 2018 12:40:04 +0000 (13:40 +0100)] 
Oops, also need import

6 months agoUse standard certificates
Sebb [Sat, 19 May 2018 12:39:06 +0000 (13:39 +0100)] 
Use standard certificates

6 months agoMay vary between installations
Sebb [Sat, 19 May 2018 12:36:09 +0000 (13:36 +0100)] 
May vary between installations

6 months agoNo longer needed (refs were dropped in c518100)
Sebb [Fri, 18 May 2018 15:05:00 +0000 (16:05 +0100)] 
No longer needed (refs were dropped in c518100)

6 months agoBug: push-failures.py expects to find 'id' key in json file
Sebb [Fri, 18 May 2018 14:44:40 +0000 (15:44 +0100)] 
Bug: push-failures.py expects to find 'id' key in json file

This fixes #454

6 months ago+=2018
Sebb [Fri, 18 May 2018 11:31:51 +0000 (12:31 +0100)] 
+=2018

6 months agoBug: ES 5.x does not support word-cloud (stats.lua)
Sebb [Thu, 17 May 2018 23:33:42 +0000 (00:33 +0100)] 
Bug: ES 5.x does not support word-cloud (stats.lua)

This fixes #345
(Actually fixed in 724837dda8c757c17c9572ef2c5a34e61a4468b9)

6 months agoAdd version info to elastic module
Sebb [Thu, 17 May 2018 23:20:34 +0000 (00:20 +0100)] 
Add version info to elastic module

6 months agoBug: setup.py fails with ES2 - fielddata
Sebb [Thu, 17 May 2018 22:58:35 +0000 (23:58 +0100)] 
Bug: setup.py fails with ES2 - fielddata

This fixes #453

6 months agoBug: setup.py --default should not prompt for urlPrefix
Sebb [Thu, 17 May 2018 15:54:38 +0000 (16:54 +0100)] 
Bug: setup.py --default should not prompt for urlPrefix

This fixes #452

6 months agoBug: copy-list.py does not work
Sebb [Thu, 17 May 2018 13:50:25 +0000 (14:50 +0100)] 
Bug: copy-list.py does not work

This fixes #450

6 months agoBug: unnecessary test (will always succeed) in copy-list.py
Sebb [Thu, 17 May 2018 12:32:30 +0000 (13:32 +0100)] 
Bug: unnecessary test (will always succeed) in copy-list.py

This fixes #451

7 months agoBug: archiver ignores failures if dumponfail is not defined
Sebb [Mon, 14 May 2018 13:02:48 +0000 (14:02 +0100)] 
Bug: archiver ignores failures if dumponfail is not defined

This fixes #449

8 months agocompuser body should be unicode
Daniel Gruno [Tue, 3 Apr 2018 11:16:02 +0000 (13:16 +0200)] 
compuser body should be unicode

This addresses #445 (though not a complete fix yet)

8 months agoAlso look for references: header when threading
Daniel Gruno [Tue, 3 Apr 2018 10:23:23 +0000 (12:23 +0200)] 
Also look for references: header when threading

Thsi fixes #444.

9 months agoEnh: make MboxoFactory optional
Sebb [Wed, 7 Mar 2018 18:22:49 +0000 (18:22 +0000)] 
Enh: make MboxoFactory optional

This fixes #442

9 months agoFix path in hash-bang
Sebb [Wed, 7 Mar 2018 01:14:43 +0000 (01:14 +0000)] 
Fix path in hash-bang

9 months agocluster generator does *not* include the lid in the hash
Sebb [Tue, 6 Mar 2018 00:53:25 +0000 (00:53 +0000)] 
cluster generator does *not* include the lid in the hash

9 months agoduplication of data in response from thread.lua
Sebb [Fri, 2 Mar 2018 20:09:35 +0000 (20:09 +0000)] 
duplication of data in response from thread.lua

This fixes #440

9 months agoOops, accidentally committed ponymail.js in a8ea8a0
Sebb [Fri, 2 Mar 2018 20:01:41 +0000 (20:01 +0000)] 
Oops, accidentally committed ponymail.js in a8ea8a0

Fix up source so js builds OK

9 months agoOops forgot to update source in 3cc594f
Sebb [Fri, 2 Mar 2018 19:55:42 +0000 (19:55 +0000)] 
Oops forgot to update source in 3cc594f

9 months agooption to reduce stats.lua output
Sebb [Thu, 1 Mar 2018 23:49:44 +0000 (23:49 +0000)] 
option to reduce stats.lua output

Additional fix for #438
Don't calculate threads if they won't be returned

9 months agoUnused
Sebb [Thu, 1 Mar 2018 23:09:59 +0000 (23:09 +0000)] 
Unused

9 months agoUse irt variable instead of refetching it
Sebb [Thu, 1 Mar 2018 23:00:16 +0000 (23:00 +0000)] 
Use irt variable instead of refetching it

9 months agostats.lua cannot handle deep nesting
Sebb [Thu, 1 Mar 2018 22:56:08 +0000 (22:56 +0000)] 
stats.lua cannot handle deep nesting

Increase max level as temporary work-round for #439

9 months agooption to reduce stats.lua output
Sebb [Thu, 1 Mar 2018 13:41:18 +0000 (13:41 +0000)] 
option to reduce stats.lua output

Additional fix for #438
Need to keep references if client is to do the threading

9 months agoDon't overwrite the original filename
Sebb [Wed, 28 Feb 2018 22:47:48 +0000 (22:47 +0000)] 
Don't overwrite the original filename

It's more useful to use the original name in most messages