Ever since Android 9+ switched to Conscrypt we can no longer efficiently
encrypt (and decrypt) large files with AES-GCM. We did’t notice this before
because when using 16 byte IVs even modern Androids will fall back to bouncy
castle. However the 'bug'/'feature' in Conscrypt surfaced when we switched over
to 12 byte IVs (which uses Conscrypt on Android 9+)
Switching back entirely to 16 byte IVs is undesirable as this would break
compatibility with Monal. So we end up with a weird compromise where we use
12 byte for normale plain text OMEMO messages and 'small' files where the
inefficiencies aren’t a problem.
The result of this commit is that Monal won’t be able to receive our files
larger than 768KiB. However the alternative is that Conversations would always
OOM when attempting to send larger files (where large depends on the available
RAM.)
fixes#3653
The Crypto provider used from Android P onwards (conscrypt) has a weird bug
that when 12 bytes IVs are used it will decrypt or encrypt the entire file
in RAM instead of streaming it. That will cause OOM for 'larger' files on http
upload. (both downloads and uploads are effected)
It is currently unclear why this is happening and why Conscrypt is put into a
different mode.
We are only observing that Android versions below P are fine and using 16 bytes
is fine on all Android versions.
usually this wasn’t a problem as this is only the fallback after no IPs
have been discovered.
this also isn‘t a security issue as worst case is the hostname doesn’t get
accepeted as fallback in cert validation.
thanks @genofire for spotting this
Daniel Gultsch
genofire