Exposing the Android platform existence to Python modules

[Shiz]

Hi folks,

I’m working on porting CPython to the Android platform, and while making decent progress, I’m currently stuck at a higher-level issue than adding #ifdefs for __ANDROID__ to C extension modules.

The idea is, not only CPython extension modules have some assumptions that don’t seem to fit Android’s mold, some default Python-written modules do as well. However, whereas CPython extensions can trivially check if we’re building for Android by checking the __ANDROID__ compiler macro, Python modules can do no such check, and are left wondering how to figure out if the platform they are currently running on is an Android one. To my knowledge there is no reliable way to detect if one is using Android as a vehicle for their journey using any other way.

Now, the main question is: what would be the best way to ‘expose’ the indication that Android is being ran on to Python-living modules? My own thought was to add sys.getlinuxuserland(), or platform.linux_userland(), in similar vein to sys.getwindowsversion() and platform.linux_distribution(), which could return information about the userland of running CPython instance, instead of knowing merely the kernel and the distribution.

This way, code could trivially check if it ran on the GNU(+associates) userland, or under a BSD-ish userland, or Android… and adjust its behaviour accordingly.

I would be delighted to hear comments on this proposal, or better yet, alternative solutions. :)

Kind regards,
Shiz

P.S.: I am well aware that Android might as well never be officially supported in CPython. In that case, consider this a thought experiment of how it /would/ be handled. :)

[Glenn Linderman]

Post by Shiz
Hi folks,
I’m working on porting CPython to the Android platform, and while making decent progress, I’m currently stuck at a higher-level issue than adding #ifdefs for __ANDROID__ to C extension modules.
The idea is, not only CPython extension modules have some assumptions that don’t seem to fit Android’s mold, some default Python-written modules do as well. However, whereas CPython extensions can trivially check if we’re building for Android by checking the __ANDROID__ compiler macro, Python modules can do no such check, and are left wondering how to figure out if the platform they are currently running on is an Android one. To my knowledge there is no reliable way to detect if one is using Android as a vehicle for their journey using any other way.
Now, the main question is: what would be the best way to ‘expose’ the indication that Android is being ran on to Python-living modules? My own thought was to add sys.getlinuxuserland(), or platform.linux_userland(), in similar vein to sys.getwindowsversion() and platform.linux_distribution(), which could return information about the userland of running CPython instance, instead of knowing merely the kernel and the distribution.

I've no idea what you mean by "userland" in your suggestions above or
below, but doesn't the Android environment qualify as a
(multi-versioned) platform independently of its host OS? Seems I've read
about an Android reimplementation for Windows, for example. As long as
all the services expected by Android are faithfully produced, the host
OS may be irrelevant to an Android application... in which case, I would
think/propose/suggest the platform name should change from win32 or
linux to Android (and the Android version be reflected in version parts).

Post by Shiz
This way, code could trivially check if it ran on the GNU(+associates) userland, or under a BSD-ish userland, or Android
 and adjust its behaviour accordingly.
I would be delighted to hear comments on this proposal, or better yet, alternative solutions. :)
Kind regards,
Shiz
P.S.: I am well aware that Android might as well never be officially supported in CPython. In that case, consider this a thought experiment of how it /would/ be handled. :)

Is your P.S. suggestive that you would not be willing to support your
port for use by others? Of course, until it is somewhat complete, it is
hard to know how complete and compatible it can be.

[Paul Moore]

I've no idea what you mean by "userland" in your suggestions above or below,
but doesn't the Android environment qualify as a (multi-versioned) platform
independently of its host OS? Seems I've read about an Android
reimplementation for Windows, for example. As long as all the services
expected by Android are faithfully produced, the host OS may be irrelevant
to an Android application... in which case, I would think/propose/suggest
the platform name should change from win32 or linux to Android (and the
Android version be reflected in version parts).

Alternatively, if having sys.platform be "linux" makes portability
easier because code that does a platform check generally gets the
right answer if Android reports as "linux", then why not make
sys.linux_distribution report "android"?

To put it briefly, either android is the platform, or android is a
specific distribution of the linux platform.

Paul

[Shiz]

Post by Paul Moore
Alternatively, if having sys.platform be "linux" makes portability
easier because code that does a platform check generally gets the
right answer if Android reports as "linux", then why not make
sys.linux_distribution report "android"?
To put it briefly, either android is the platform, or android is a
specific distribution of the linux platform.
Paul

That might maybe work better. I was assuming a userland perspective because I’ve been honestly mostly wrestling with Bionic, Android’s libc,
but putting that into perspective to consider Android as a whole (after all, the SDK and NDK are what make Android for a lot of developers)
might be a valid other approach as well.

Kinds regards,
Shiz

[Shiz]

I've no idea what you mean by "userland" in your suggestions above or below, but doesn't the Android environment qualify as a (multi-versioned) platform independently of its host OS? Seems I've read about an Android reimplementation for Windows, for example. As long as all the services expected by Android are faithfully produced, the host OS may be irrelevant to an Android application... in which case, I would think/propose/suggest the platform name should change from win32 or linux to Android (and the Android version be reflected in version parts).

That might be a way to look at it. So far I assumed that the Android environment would be largely Linux-based, since the Android NDK (Native Development Kit, the SDK used for creating C/C++-level applications) is used for my patch which gives a GNU-ish toolchain with a Linux/Unixy environment. I know an implementation exists that claims to run Android on top of an NT kernel, but I honestly have little idea of how it works. Given how a fair amount of things ‘already work’ with the platform set to linux, I’m not sure if changing sys.platform would be a good idea… but that’s from my NDK perspective.

Is your P.S. suggestive that you would not be willing to support your port for use by others? Of course, until it is somewhat complete, it is hard to know how complete and compatible it can be.

Oh, no, nothing like that. It’s just that I’m not sure, as goes for anything, that it would be accepted into mainline CPython. Better safe than sorry in that aspect: maybe the maintainers don’t want to support Android in the first place. :)

Kind regards,
Shiz

[Charles-François Natali]

Is your P.S. suggestive that you would not be willing to support your port for use by others? Of course, until it is somewhat complete, it is hard to know how complete and compatible it can be.

Oh, no, nothing like that. It's just that I'm not sure, as goes for anything, that it would be accepted into mainline CPython. Better safe than sorry in that aspect: maybe the maintainers don't want to support Android in the first place. :)

Well, Android is so popular that supporting it would definitely be interesting.
There are a couple questions however (I'm not familiar at all with
Android, I don't have a smartphone ;-):
- Do you have an idea of the amount of work/patch size required? Do
you have an example of a patch (even if it's a work-in-progess)?
- Is there really a common Android platform? I've heard a lot about
fragmentation, so would we have to support several Android flavours
(like #ifdef __ANDROID_VENDOR_A__, #elif defined
__ANDROID_VENDOR_B__)?

[Shiz]

Post by Charles-François Natali
Well, Android is so popular that supporting it would definitely be
interesting. There are a couple questions however (I'm not familiar
at all with Android, I don't have a smartphone ;-): - Do you have an
idea of the amount of work/patch size required? Do you have an
example of a patch (even if it's a work-in-progess)? - Is there
really a common Android platform? I've heard a lot about
fragmentation, so would we have to support several Android flavours
(like #ifdef __ANDROID_VENDOR_A__, #elif defined
__ANDROID_VENDOR_B__)?

Absolutely! I maintain a public repository of patches against CPython
v3.3.3 at [1].

They are divided into three large patches: one fixes some issues I
encountered with CPython's build system for cross-compilation, one fixes
Android/Bionic's numerous locale issues (locale.h/langinfo.h support in
Android is basically a set of stub functions that return NULL), and the
last one is a set of 'misc' fixes for things that affect Android, mainly
smaller things like missing fields in struct passwd and the like.

With those patches, CPython 3.3.3 will cross-compile to and run on at
least my own Android device, a Moto G running Android 4.4.2. What's left
to fail is fix the numerous regression test failures and their causes. I
documented some of my findings at [2]. :)

As far as Android fragmentation goes, to my knowledge that mainly refers
to fragmentation at two levels: the Android versions numerous devices
run tends to differ greatly, and the screen sizes, resolutions and
aspect ratios vary greatly. Obviously the latter is a problem beyond the
scope of CPython, but the former could lead to some issues.

Luckily however, the NDK[3], the SDK of choice to use for C/C++-level
applications, is fairly unified and expected to be used for pretty much
all Android devices. Essentially there should be only the NDK for
CPython to target, with a variety of NDK versions to support depending
on which versions of Android CPython chooses to support. So far I've
been only testing against NDK r9c, so I'm honestly not all that familiar
with the changes different NDK versions bring, but from what I heard up
until NDK r10 were mostly toolchain updates and header additions for new
Android versions.

I'd dare say that the vast, vast majority of Android devices out there
are running on the same base, namely AOSP[4] with numerous vendor
fixes/drivers/additions, and that custom Android distributions would try
not to break NDK compatibility.

Kind regards,
Shiz

[1]: https://github.com/rave-engine/python3-android/tree/master/src
[2]: https://github.com/rave-engine/python3-android/issues/3
[3]: https://developer.android.com/tools/sdk/ndk/index.html
[4]: https://source.android.com/

[Akira Li]

This post might be inappropriate. Click to display it.

[Steven D'Aprano]

Post by Akira Li
Python uses os.name, sys.platform, and various functions from `platform`

[...]

Post by Akira Li
If Android is posixy enough (would `posix` module work on Android?)
then os.name could be left 'posix'.

Does anyone know what kivy does when running under Android?

[Guido van Rossum]

Or SL4A? (https://github.com/damonkohler/sl4a)

Post by Steven D'Aprano

Post by Akira Li
Python uses os.name, sys.platform, and various functions from `platform`

[...]

Post by Akira Li
If Android is posixy enough (would `posix` module work on Android?)
then os.name could be left 'posix'.

Does anyone know what kivy does when running under Android?
--
Steven
_______________________________________________
Python-Dev mailing list
https://mail.python.org/mailman/listinfo/python-dev
https://mail.python.org/mailman/options/python-dev/guido%40python.org