[Mayan EDMS: 2178] mayan performance in Azure Cloud (managed)

[Tony Nys]

For anyone interested.
I've installed Mayan into a managed Azure cloud environment:
* Postgres managed Azure database (basic configuration)
* Documents stored in Azure blob storage
* mayan VM itself in a Ubuntu 2core 8GB configuration

So no data (except cache data/logfiles) are stored inside this "worker vm"
which can be scaled behind a load balancer.

When testing, Mayan was extremely slow.
* Default mayan uwsgi setup is 1 process , 1 Thread => too slow, timeouts,
etc.. and database/VM idle most of the time
=> fixed by changing uwsg.ini: added processes & threads parameters
* Connection pooling in Django was not set up: added parameter
"CONN_MAX_AGE 500" in the local.py "DATABASES" variable
* database & VM & blobstorage were not in same Azure region; => moved them
to same region

Performance tests were all done with the Mayan REST api & Jmeter at client
side
Perfomance results were then much better, expect for "cabinet" operations :
these were unacceptable/unusable; so advice not to use them



Note
* that the API swagged doc is not clear, reverse engineering needed to find
working api call examples
* mayan should install by default "uwsgitop" command in image
* threads/process in uwsg.ini should be higher by default: I've used 4
processes, 2 threads
see image below for results: see Column "O" for the tweaked process/thread
configuration inside uwsgi,
compared to column "N" in the mayan/uwsgi default configuration (1 process,
1 thread)




<Loading Image...>

[Tony Nys]

note: there should be a new AP method / feature to
* when posting documents: add metadatas when posting; now for 1 doc with 3
metadata, we need 4 api calls
* when posting document: include version id & 1st page id
* get 1st page image of a document ID; now we need to search document; then
get versions/pages and then we now the page id

Also have example docs for the APIs, especially search is not clear

Post by Tony Nys
For anyone interested.
* Postgres managed Azure database (basic configuration)
* Documents stored in Azure blob storage
* mayan VM itself in a Ubuntu 2core 8GB configuration
So no data (except cache data/logfiles) are stored inside this "worker vm"
which can be scaled behind a load balancer.
When testing, Mayan was extremely slow.
* Default mayan uwsgi setup is 1 process , 1 Thread => too slow, timeouts,
etc.. and database/VM idle most of the time
=> fixed by changing uwsg.ini: added processes & threads parameters
* Connection pooling in Django was not set up: added parameter
"CONN_MAX_AGE 500" in the local.py "DATABASES" variable
* database & VM & blobstorage were not in same Azure region; => moved them
to same region
Performance tests were all done with the Mayan REST api & Jmeter at client
side
Perfomance results were then much better, expect for "cabinet" operations
: these were unacceptable/unusable; so advice not to use them
Note
* that the API swagged doc is not clear, reverse engineering needed to
find working api call examples
* mayan should install by default "uwsgitop" command in image
* threads/process in uwsg.ini should be higher by default: I've used 4
processes, 2 threads
see image below for results: see Column "O" for the tweaked process/thread
configuration inside uwsgi,
compared to column "N" in the mayan/uwsgi default configuration (1
process, 1 thread)
<https://lh3.googleusercontent.com/-kwo49FLHuOw/WfHWF0cc8aI/AAAAAAAAAXE/VnPoQXDpb6UYuoiMCq2DYJyjFyi6BUBLwCLcBGAs/s1600/Mayan_Perf_Azure_Postgres_Blobstorage.png>