Discussion:
Software integrators vs end users (was Re: Language Summit notes)
(too old to reply)
Nick Coghlan
2014-04-18 15:58:59 UTC
Permalink
There is an 'Installing Python on Windows' link further down the google
results that links to a fairly good page from python-guide.org, whose
first link lets you download the 2.7.6 msi. I guess that's the 32
bit version. But it then tells you go to python.org to make sure you
get the latest release, and tells you to click on a link that doesn't
exist any more (the "windows installer" link).
So, yeah.
Usability.
As part of thrashing out the respective distribution ecosystem roles
of pip and conda (still a work in progress), we're at least converging
on the notion that there are actually now *two* main ways of consuming
Python: as a "software integrator" (the way most of us have
traditionally consumed it, and the way that dominates most project
documentation outside the scientific Python community) and as an "end
user" (the way Linux system administrators have long consumed it, and
the way scientists, financial analysts and folks just learning Python
are likely best off consuming it).

Making these different personas explicit is a process that has barely
begun (this email is mostly based on some conversations I had in
person at PyCon and via email during the sprints), but here's the gist
(based on listing examples):

Software integrators:

* Linux distributions and other operating system vendors
* Sumo redistributions (commercial or otherwise)
* "Python based environments" (PTVS, Enthought Canopy, wakari.io,
Python Anywhere, etc)
* Software-as-a-Service developers
* Device manufacturers
* PC OEMs
* creators of corporate "Standard Operating Environment" definitions
* System integrators (IBM, Boeing et al)
* Application developers (from simple CLI tools to OpenStack)

End users:
* system administrators
* scientists (whether in academia or corporations)
* financial analysts
* engineers
* data miners
* graphic artists
* animation designers
* <<< Folks trying out Python for the first time >>>

For end users, Python is likely consumed as *part of something else*.
For Linux admins, it's a way of scripting the system, for graphic
artists and animators, it's likely embedded as part of a tool like
Blender or Maya, for scientists, financial analysts, engineers and
data miners, it likely makes sense to use it as part of a larger
visualisation environment like IPython Notebook, Python (x, y),
Enthought Canopy, or a hosted system like Wakari or Python Anywhere.

Folks just learning Python are also in the latter list, and we
currently ask them (on the python.org download pages) to jump straight
into the "software integrator" role to get their environment set up,
rather than setting out to impress them by recommending one of the
pre-integrated sumo distributions that offers quick and easy access to
powerful visualisation and data analysis tools. While providing access
to the CPython default interactive prompt in the browser is cool, it's
less impressive as the default experience we provide after someone has
downloaded and installed the interpreter. Instead, we likely want to
*really* impress them by making it easy for them to explore the full
power of things like IPython Notebooks. My own current preferred
approach for that is the fully open source "Anaconda" distribution
from Continuum Analytics, specifically because it *is* fully open
source, and you can "pip install conda" to bootstrap their package
manager in other contexts. And, importantly, because conda
environments can *manage the Python runtime itself*, it is able to
work more consistently across different platforms than the upstream
tools by reliably isolating itself from the system Python on POSIX
platforms. It should even by possible to get conda to manage alternate
Python implementations like Stackless, PyPy, Jython, IronPython, etc
(although I don't believe anyone is using it that way as yet).

However, conda *isn't* optimised for the software integrator use case
- the additional work it does to improve cross platform consistency
actually *gets in the way*, when you're trying to integrate Python
closely with a larger system (like a Linux distribution), and "a large
set of automatically provided libraries" isn't actually a feature in
that context. While conda does offer some options (like miniconda) to
make that kind of integration task easier, I strongly believe that
realm is better handled by consuming CPython and the standard library,
along with pip and related tools, directly, and accepting the
additional platform specific details that come along with that.

Making this "software integrator" and "end user" split deliberately
and consciously, and pushing the former toward consuming things in a
more DIY fashion, and the latter towards a pre-assembled sumo
distribution should help greatly in allowing us to better balance the
starkly different needs of the two groups, and provide an extremely
high quality "out of the box" experience for new users, while still
allowing software integrators to dive in and customise things to suit
their own needs.
From a Python 2->3 migration perspective, blessing at least one sumo
distribution also provides a way to make it easy to gain ready access
to backports from the Python 3 standard library (for example, Anaconda
currently includes at least the backports.ssl_match_hostname module).

Regards,
Nick.
Guido van Rossum
2014-04-18 16:31:04 UTC
Permalink
Could I summarize that as "software integrators build from source, while
end users use an installer"? And the rest of the discussion is about which
installer (in the widest sense of the word) to recommend, where the choices
include Linux vendor distros, sumo Python distros, Mac/Win installers, as
well as fine-grained tools like apt-get (Linux vendor specific) and pip and
conda (Python specific)? And perhaps conda is pushing the boundaries
because it also tries to manage things that aren't strictly Python, but
useful for scientists?
Post by Nick Coghlan
There is an 'Installing Python on Windows' link further down the google
results that links to a fairly good page from python-guide.org, whose
first link lets you download the 2.7.6 msi. I guess that's the 32
bit version. But it then tells you go to python.org to make sure you
get the latest release, and tells you to click on a link that doesn't
exist any more (the "windows installer" link).
So, yeah.
Usability.
As part of thrashing out the respective distribution ecosystem roles
of pip and conda (still a work in progress), we're at least converging
on the notion that there are actually now *two* main ways of consuming
Python: as a "software integrator" (the way most of us have
traditionally consumed it, and the way that dominates most project
documentation outside the scientific Python community) and as an "end
user" (the way Linux system administrators have long consumed it, and
the way scientists, financial analysts and folks just learning Python
are likely best off consuming it).
Making these different personas explicit is a process that has barely
begun (this email is mostly based on some conversations I had in
person at PyCon and via email during the sprints), but here's the gist
* Linux distributions and other operating system vendors
* Sumo redistributions (commercial or otherwise)
* "Python based environments" (PTVS, Enthought Canopy, wakari.io,
Python Anywhere, etc)
* Software-as-a-Service developers
* Device manufacturers
* PC OEMs
* creators of corporate "Standard Operating Environment" definitions
* System integrators (IBM, Boeing et al)
* Application developers (from simple CLI tools to OpenStack)
* system administrators
* scientists (whether in academia or corporations)
* financial analysts
* engineers
* data miners
* graphic artists
* animation designers
* <<< Folks trying out Python for the first time >>>
For end users, Python is likely consumed as *part of something else*.
For Linux admins, it's a way of scripting the system, for graphic
artists and animators, it's likely embedded as part of a tool like
Blender or Maya, for scientists, financial analysts, engineers and
data miners, it likely makes sense to use it as part of a larger
visualisation environment like IPython Notebook, Python (x, y),
Enthought Canopy, or a hosted system like Wakari or Python Anywhere.
Folks just learning Python are also in the latter list, and we
currently ask them (on the python.org download pages) to jump straight
into the "software integrator" role to get their environment set up,
rather than setting out to impress them by recommending one of the
pre-integrated sumo distributions that offers quick and easy access to
powerful visualisation and data analysis tools. While providing access
to the CPython default interactive prompt in the browser is cool, it's
less impressive as the default experience we provide after someone has
downloaded and installed the interpreter. Instead, we likely want to
*really* impress them by making it easy for them to explore the full
power of things like IPython Notebooks. My own current preferred
approach for that is the fully open source "Anaconda" distribution
from Continuum Analytics, specifically because it *is* fully open
source, and you can "pip install conda" to bootstrap their package
manager in other contexts. And, importantly, because conda
environments can *manage the Python runtime itself*, it is able to
work more consistently across different platforms than the upstream
tools by reliably isolating itself from the system Python on POSIX
platforms. It should even by possible to get conda to manage alternate
Python implementations like Stackless, PyPy, Jython, IronPython, etc
(although I don't believe anyone is using it that way as yet).
However, conda *isn't* optimised for the software integrator use case
- the additional work it does to improve cross platform consistency
actually *gets in the way*, when you're trying to integrate Python
closely with a larger system (like a Linux distribution), and "a large
set of automatically provided libraries" isn't actually a feature in
that context. While conda does offer some options (like miniconda) to
make that kind of integration task easier, I strongly believe that
realm is better handled by consuming CPython and the standard library,
along with pip and related tools, directly, and accepting the
additional platform specific details that come along with that.
Making this "software integrator" and "end user" split deliberately
and consciously, and pushing the former toward consuming things in a
more DIY fashion, and the latter towards a pre-assembled sumo
distribution should help greatly in allowing us to better balance the
starkly different needs of the two groups, and provide an extremely
high quality "out of the box" experience for new users, while still
allowing software integrators to dive in and customise things to suit
their own needs.
From a Python 2->3 migration perspective, blessing at least one sumo
distribution also provides a way to make it easy to gain ready access
to backports from the Python 3 standard library (for example, Anaconda
currently includes at least the backports.ssl_match_hostname module).
Regards,
Nick.
_______________________________________________
Python-Dev mailing list
https://mail.python.org/mailman/listinfo/python-dev
https://mail.python.org/mailman/options/python-dev/guido%40python.org
--
--Guido van Rossum (python.org/~guido)
Paul Moore
2014-04-18 16:55:02 UTC
Permalink
Post by Nick Coghlan
As part of thrashing out the respective distribution ecosystem roles
of pip and conda (still a work in progress), we're at least converging
on the notion that there are actually now *two* main ways of consuming
Python: as a "software integrator" (the way most of us have
traditionally consumed it, and the way that dominates most project
documentation outside the scientific Python community) and as an "end
user" (the way Linux system administrators have long consumed it, and
the way scientists, financial analysts and folks just learning Python
are likely best off consuming it).
Making these different personas explicit is a process that has barely
begun (this email is mostly based on some conversations I had in
person at PyCon and via email during the sprints), but here's the gist
Interesting perspective. However, I'm not convinced it's complete.
Specifically, there's one group of people who I encounter relatively
often, who don't seem to me to fit well into either category you're
proposing. That is, (Windows in my experience, but maybe Linux as
well) users who want to write "simple scripts" and for whom batch
files or similar are not sufficient. Such people typically don't have
the sort of "single application area" focus that your "end user"
category seems to imply, but on the other hand they don't really fit
the "software integrators" role in the sense of necessarily being
comfortable setting up their own development environment.

I worry that your classification risks ignoring that group (maybe
because Unix users are well served with other alternatives than Python
for this type of task, or because on Unix "use the system Python" is
the right answer).

Your list of "end user" targeted distributions seem to be limited to:

- Linux distribution vendors
- Vendors focused on the essentially scientific community (in the
broadest sense)
- Embedded Python

That's very far from being complete coverage of all the people *I'd*
like to be able to recommend Python to. Specifically, unless we're not
interested in "generic" Windows users, I think we need to offer *some*
form of equivalent of the OS-packaged Python on Linux for Windows
users. That's what the python.org builds, plus pip, wheels and PyPI,
give for Windows users now. Hmm, if we assume that supporting that
remains a priority, is what you're really saying that we *don't* try
to extend that to work for Linux/OSX, as doing so competes with the OS
vendors - but rather we see python.org binaries and binary
infrastructure like wheels as being focused on the Windows user
experience?

(I wish I'd been at PyCon, this would have been a very interesting
discussion to have face to face. Email isn't ideal for this...)

Paul.
Daniel Holth
2014-04-18 17:14:33 UTC
Permalink
Post by Paul Moore
Post by Nick Coghlan
As part of thrashing out the respective distribution ecosystem roles
of pip and conda (still a work in progress), we're at least converging
on the notion that there are actually now *two* main ways of consuming
Python: as a "software integrator" (the way most of us have
traditionally consumed it, and the way that dominates most project
documentation outside the scientific Python community) and as an "end
user" (the way Linux system administrators have long consumed it, and
the way scientists, financial analysts and folks just learning Python
are likely best off consuming it).
Making these different personas explicit is a process that has barely
begun (this email is mostly based on some conversations I had in
person at PyCon and via email during the sprints), but here's the gist
Interesting perspective. However, I'm not convinced it's complete.
Specifically, there's one group of people who I encounter relatively
often, who don't seem to me to fit well into either category you're
proposing. That is, (Windows in my experience, but maybe Linux as
well) users who want to write "simple scripts" and for whom batch
files or similar are not sufficient. Such people typically don't have
the sort of "single application area" focus that your "end user"
category seems to imply, but on the other hand they don't really fit
the "software integrators" role in the sense of necessarily being
comfortable setting up their own development environment.
I worry that your classification risks ignoring that group (maybe
because Unix users are well served with other alternatives than Python
for this type of task, or because on Unix "use the system Python" is
the right answer).
- Linux distribution vendors
- Vendors focused on the essentially scientific community (in the
broadest sense)
- Embedded Python
That's very far from being complete coverage of all the people *I'd*
like to be able to recommend Python to. Specifically, unless we're not
interested in "generic" Windows users, I think we need to offer *some*
form of equivalent of the OS-packaged Python on Linux for Windows
users. That's what the python.org builds, plus pip, wheels and PyPI,
give for Windows users now. Hmm, if we assume that supporting that
remains a priority, is what you're really saying that we *don't* try
to extend that to work for Linux/OSX, as doing so competes with the OS
vendors - but rather we see python.org binaries and binary
infrastructure like wheels as being focused on the Windows user
experience?
(I wish I'd been at PyCon, this would have been a very interesting
discussion to have face to face. Email isn't ideal for this...)
One more group that I find interesting is application users. These
people should not need to notice that Python is in use at all, in
contrast to the "build a virtualenv / install / pass through fire and
death / use" workflow that is sometimes promoted. They are well served
by good tools that make single-file zip distributions or py2exe etc.
from a collection of wheels or sdists. Application users are using
Python because a best-in-class program is written in Python and not
because a Python program integrates better with other Python
libraries.


Conda is interesting because it is a system package manager, except it
installs everything into what they sometimes describe as "C-level
virtualenvs" rather than /. They've come up with a design that works
well with Python programs but isn't particularly Python-specific at
all.
Nick Coghlan
2014-04-18 19:18:36 UTC
Permalink
Post by Paul Moore
Post by Nick Coghlan
As part of thrashing out the respective distribution ecosystem roles
of pip and conda (still a work in progress), we're at least converging
on the notion that there are actually now *two* main ways of consuming
Python: as a "software integrator" (the way most of us have
traditionally consumed it, and the way that dominates most project
documentation outside the scientific Python community) and as an "end
user" (the way Linux system administrators have long consumed it, and
the way scientists, financial analysts and folks just learning Python
are likely best off consuming it).
Making these different personas explicit is a process that has barely
begun (this email is mostly based on some conversations I had in
person at PyCon and via email during the sprints), but here's the gist
Interesting perspective. However, I'm not convinced it's complete.
Personas are never complete - however, they represent sufficiently
useful archetypes that help guide design decisions. Where alarm bells
need to start ringing is when two personas are far enough apart in
interests and skill levels that it is highly unlikely that one tool
will be able to satisfy both groups of users.

I think that's the situation we find ourselves in with respect to
folks that are already professional programmers and folks that just
need to do some programming as part of another activity.
Post by Paul Moore
Specifically, there's one group of people who I encounter relatively
often, who don't seem to me to fit well into either category you're
proposing. That is, (Windows in my experience, but maybe Linux as
well) users who want to write "simple scripts" and for whom batch
files or similar are not sufficient. Such people typically don't have
the sort of "single application area" focus that your "end user"
category seems to imply, but on the other hand they don't really fit
the "software integrators" role in the sense of necessarily being
comfortable setting up their own development environment.
The "single application area" in that case is "Windows scripting", and
it *would* make sense to have a sumo distribution that provided
PyWin32 and other components. That distribution may include other
things as well, though.
Post by Paul Moore
I worry that your classification risks ignoring that group (maybe
because Unix users are well served with other alternatives than Python
for this type of task, or because on Unix "use the system Python" is
the right answer).
I think the conda "end user" approach handles that case pretty well
(while it certainly started with the scientific Python stack, conda
certainly isn't limited to that). So the entry can be "system
administrators" in general, rather than being specific to Linux.

The "lack of integration" issues on Linux and Mac OS X show up because
there *is* a system Python for conda to be separate from. We don't
have that problem on Windows (although integrating with the Python
launcher may be an issue).
Post by Paul Moore
- Linux distribution vendors
As noted above, this category is broader than just Linux - it really
covers any system scripting context.
Post by Paul Moore
- Vendors focused on the essentially scientific community (in the
broadest sense)
I'd say "analytical", rather than specifically "scientific". From a
user experience perspective though, the important point is that many
of the folks being targeted *aren't* programmers in the traditional
sense - they just need to do some programming as part of their actual
job. It turns out that's also a useful usability proxy for people just
starting out with programming :)
Post by Paul Moore
- Embedded Python
That's very far from being complete coverage of all the people *I'd*
like to be able to recommend Python to. Specifically, unless we're not
interested in "generic" Windows users,
I think conda covers that "generic Windows user" case as well - that's
why I emphasised the fully open source nature of it. We've also been
having some interesting conversations with Travis about potentially
improving the level of collaboration between the PyPA and conda devs.
Post by Paul Moore
I think we need to offer *some*
form of equivalent of the OS-packaged Python on Linux for Windows
users. That's what the python.org builds, plus pip, wheels and PyPI,
give for Windows users now. Hmm, if we assume that supporting that
remains a priority, is what you're really saying that we *don't* try
to extend that to work for Linux/OSX, as doing so competes with the OS
vendors - but rather we see python.org binaries and binary
infrastructure like wheels as being focused on the Windows user
experience?
I'd still like to improve the wheel story for Linux (there are various
benefits of that which accrue to system integrators as well), but I
think the key benefit of promoting both pip *and* conda is that it
lets us hit *two* different points on the flexibility vs usability
curve, rather than being limited to one.

pip needs to preserve a lot of flexibility, because it's already the
upstream for a *lot* of different use cases. However, that flexibility
often comes at a cost in usability, especially for new users.

Since conda *doesn't* play that upstream role the way pip does, the
flexibility can be dialled down a bit in favour of increased "out of
the box" usability, making it easier to offer a compelling new user
experience.

Ideally we'd like to make the two tools share a common back end to
reduce duplicated effort (and distlib may end up filling that role),
but I think such a split makes a lot of sense from a front end
perspective.
Post by Paul Moore
(I wish I'd been at PyCon, this would have been a very interesting
discussion to have face to face. Email isn't ideal for this...)
We didn't even get far with it at PyCon - there's certainly no
consensus around these characterisations at this point. The discussion
was mostly focused on pip vs conda, since the requirements for the two
tools are very, very similar. My contention is that it *doesn't* make
sense to try to make conda a replacement for pip (or vice-versa)
because the two tools are focused on different *audiences*, even
though an awful lot of the backend engineering requirements are the
same.

At this point, however, I'm mainly looking for consensus that there
*are* two different problems to be solved here, and solving them both
well in a single tool is likely to be nigh on impossible. (I'm aware
we're really on the wrong list for that discussion, but I also think
there's value in getting some broader python-dev awareness of this
particular topic)

Regards,
Nick.
Post by Paul Moore
Paul.
--
Nick Coghlan | ***@gmail.com | Brisbane, Australia
Donald Stufft
2014-04-18 19:39:13 UTC
Permalink
Post by Nick Coghlan
At this point, however, I'm mainly looking for consensus that there
*are* two different problems to be solved here, and solving them both
well in a single tool is likely to be nigh on impossible. (I'm aware
we're really on the wrong list for that discussion, but I also think
there's value in getting some broader python-dev awareness of this
particular topic)
I’m not sure about this? I mean yes those are two different areas, but I’m
not sure about the split between Conda and pip here. As far as I can tell
Conda is useful in the same way apt-get or yum is useful, you get a non
Python specific set of packages (because sometimes things aren’t pure
python) and you also remove a little bit of thinking about versions (although
honestly I think it’s possible to remove most of that for consumers of
packages).

To be quite frank, a lot of the benefit of Conda outside of the “I need something
that isn’t strictly Python) is in the fact they can bootstrap compiled packages
whereas pip/wheel/PyPI combination we need to convince authors to upload
their own binary packages (or at least develop something to make it easier
like a build farm).

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Nick Coghlan
2014-04-18 20:22:49 UTC
Permalink
Post by Nick Coghlan
At this point, however, I'm mainly looking for consensus that there
*are* two different problems to be solved here, and solving them both
well in a single tool is likely to be nigh on impossible. (I'm aware
we're really on the wrong list for that discussion, but I also think
there's value in getting some broader python-dev awareness of this
particular topic)
I’m not sure about this? I mean yes those are two different areas, but I’m
not sure about the split between Conda and pip here. As far as I can tell
Conda is useful in the same way apt-get or yum is useful, you get a non
Python specific set of packages (because sometimes things aren’t pure
python) and you also remove a little bit of thinking about versions (although
honestly I think it’s possible to remove most of that for consumers of
packages).
You also get the ability to use that same system to update *Python
itself*, regardless of platform. Being notified of and consuming
CPython updates on Windows, as well as consuming alternate versions on
Linux distros, is currently not a well solved problem - with conda,
it's no more complex than updating a PyPI package.

That's one of the most attractive aspects for me - making Python 3.4,
pypy, etc as easy to update with consistent cross platform
instructions as Python packages are.
To be quite frank, a lot of the benefit of Conda outside of the “I need something
that isn’t strictly Python) is in the fact they can bootstrap compiled packages
whereas pip/wheel/PyPI combination we need to convince authors to upload
their own binary packages (or at least develop something to make it easier
like a build farm).
Yep, that's a large part of why I think "divide and conquer" is the
way to go here. While it isn't completely accurate (as most SaaS
developers don't want to build C extensions from source) I think
Guido's "build from source" vs "install a pre-built binary"
distinction is still a reasonably good way to characterise it. For a
distro like Fedora (or, even more so, RHEL), we're not going to trust
a binary created by someone else if we can possibly avoid it, so
upstream binaries aren't useful to us, but pip's ability to abstract
away the vagaries of the upcoming metadata 2.0 migration is incredibly
helpful. The other distros are in the same situation (we'll always
feed source tarballs into our own build systems), and ditto for the
conda folks. We need that for all sorts of reasons that potential new
Python users don't care about, but continuing to meet our
requirements, along with the free-for-all that is PyPI makes handling
the binary distribution problem *much* harder for pip.

By contrast, like any other distro, conda doesn't need to boil the
ocean - it just needs to provide a filtered, up to date set of core
packages that work well together. The advantage it has over other
distros is that it is *cross-platform* - it works essentially the same
way on Windows, Mac OS X. Most other package management systems are
either platform specific or can't handle arbitrary binary
dependencies. By being Python-centric (even if not Python specific),
there's also a strong focus on updating the core packages more often
than the Linux distros do.

There's no "always use conda instead of pip" competition here - we
need pip, we need wheels. But I see those as software integrator
focused tools that would need a lot of additional work to provide a
truly spectacularly compelling out of the box experience for new
users. I don't think that's a useful way to expend effort - I think it
makes more sense to work on a separate "here's the fully assembled
environment for using Python as a tool to develop ideas" introduction,
while pushing the "here's how to build your own custom environment
from upstream parts" as a later step in a new user's journey towards
full Python mastery (if they're interested in that path). Making sure
that "pip install foo" does the right thing inside conda environments
(if it doesn't already) should be all that is needed to ensure that
random installation instructions on the internet still work.

Cheers,
Nick.
--
Nick Coghlan | ***@gmail.com | Brisbane, Australia
Donald Stufft
2014-04-18 20:50:44 UTC
Permalink
Post by Nick Coghlan
Post by Donald Stufft
Post by Nick Coghlan
At this point, however, I'm mainly looking for consensus that there
*are* two different problems to be solved here, and solving them both
well in a single tool is likely to be nigh on impossible. (I'm aware
we're really on the wrong list for that discussion, but I also think
there's value in getting some broader python-dev awareness of this
particular topic)
I’m not sure about this? I mean yes those are two different areas, but I’m
not sure about the split between Conda and pip here. As far as I can tell
Conda is useful in the same way apt-get or yum is useful, you get a non
Python specific set of packages (because sometimes things aren’t pure
python) and you also remove a little bit of thinking about versions (although
honestly I think it’s possible to remove most of that for consumers of
packages).
You also get the ability to use that same system to update *Python
itself*, regardless of platform. Being notified of and consuming
CPython updates on Windows, as well as consuming alternate versions on
Linux distros, is currently not a well solved problem - with conda,
it's no more complex than updating a PyPI package.
That's one of the most attractive aspects for me - making Python 3.4,
pypy, etc as easy to update with consistent cross platform
instructions as Python packages are.
Ah right. I would never want that personally so I forgot about it. (I also don’t use
the system Python for development but I user a different tool for managing it).
Post by Nick Coghlan
Post by Donald Stufft
To be quite frank, a lot of the benefit of Conda outside of the “I need something
that isn’t strictly Python) is in the fact they can bootstrap compiled packages
whereas pip/wheel/PyPI combination we need to convince authors to upload
their own binary packages (or at least develop something to make it easier
like a build farm).
Yep, that's a large part of why I think "divide and conquer" is the
way to go here. While it isn't completely accurate (as most SaaS
developers don't want to build C extensions from source) I think
Guido's "build from source" vs "install a pre-built binary"
distinction is still a reasonably good way to characterise it. For a
distro like Fedora (or, even more so, RHEL), we're not going to trust
a binary created by someone else if we can possibly avoid it, so
upstream binaries aren't useful to us, but pip's ability to abstract
away the vagaries of the upcoming metadata 2.0 migration is incredibly
helpful. The other distros are in the same situation (we'll always
feed source tarballs into our own build systems), and ditto for the
conda folks. We need that for all sorts of reasons that potential new
Python users don't care about, but continuing to meet our
requirements, along with the free-for-all that is PyPI makes handling
the binary distribution problem *much* harder for pip.
By contrast, like any other distro, conda doesn't need to boil the
ocean - it just needs to provide a filtered, up to date set of core
packages that work well together. The advantage it has over other
distros is that it is *cross-platform* - it works essentially the same
way on Windows, Mac OS X. Most other package management systems are
either platform specific or can't handle arbitrary binary
dependencies. By being Python-centric (even if not Python specific),
there's also a strong focus on updating the core packages more often
than the Linux distros do.
There's no "always use conda instead of pip" competition here - we
need pip, we need wheels. But I see those as software integrator
focused tools that would need a lot of additional work to provide a
truly spectacularly compelling out of the box experience for new
users. I don't think that's a useful way to expend effort - I think it
makes more sense to work on a separate "here's the fully assembled
environment for using Python as a tool to develop ideas" introduction,
while pushing the "here's how to build your own custom environment
from upstream parts" as a later step in a new user's journey towards
full Python mastery (if they're interested in that path). Making sure
that "pip install foo" does the right thing inside conda environments
(if it doesn't already) should be all that is needed to ensure that
random installation instructions on the internet still work.
So I’m not really worried about a competition or anything. I’m mostly worried
about confusion of users. What you’re suggestion we give to use is *two* ways
to install Python packages (and 2 or 3 ways to virtualize a Python instance).
That provides extra cognitive burden for people who are new to Python. They
have to both understand that the packages you install from Conda are different
than the ones you install from pip, and that they come from different places.
If a new user is reasonably likely to have to use ``pip install`` when they are
using pip, then I think that provides a worse experience than only having to
use one tool to manage your packages. This confusion is going to be worse when
new users find a library they want to use and it tells them to use
``pip install`` (or even easy_install!) even if Anaconda itself has a package
inside of it.

So I think this could potentially in the short term make things easier for
people who need nothing but what conda provides (because of the bootstrapped
pre-compiled binaries), but I think ultimately you’re going to confuse them
until they know enough about the ecosystem to grasp the difference.

I’ve *personally* seen this first hand with the Linux and other OSs who
distribute stuff where people get really confused about what the difference is
between pip and apt-get/yum/portmaster and when they’d use one over the other,
adding another column in the matrix for new users seems to be something that is
only going to cause more confusion and pain.

As you said the difference is often "precompiled vs source: and I think the
answer to that is to make it easier to use precompiled stuff in pip and PyPI
and not to push people to an entirely different stack and invalidate the vast
bulk of documentation (which most of it already recommends pip) that those
users are likely to encounter.

Baring implementation problems (bad defaults, ux, etc) the fundamental
difference between conda and pip/easy_install is that conda manages more than
just Python packages, much like apt-get/yum) does and any discussion about what
to recommend where should focus on fundamental differences as implementation
problems in *either* solution can be addressed (lack of binary/ux on pip,
security issues on Conda, etc).

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Donald Stufft
2014-04-18 20:51:49 UTC
Permalink
Post by Donald Stufft
So I’m not really worried about a competition or anything. I’m mostly worried
about confusion of users. What you’re suggestion we give to use is *two* ways
to install Python packages (and 2 or 3 ways to virtualize a Python instance).
That provides extra cognitive burden for people who are new to Python. They
have to both understand that the packages you install from Conda are different
than the ones you install from pip, and that they come from different places.
If a new user is reasonably likely to have to use ``pip install`` when they are
using pip, then I think that provides a worse experience than only having to
use one tool to manage your packages. This confusion is going to be worse when
new users find a library they want to use and it tells them to use
``pip install`` (or even easy_install!) even if Anaconda itself has a package
inside of it.
Typo - When they are using conda.
-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Nick Coghlan
2014-04-18 21:08:56 UTC
Permalink
So I’m not really worried about a competition or anything. I’m mostly worried
about confusion of users. What you’re suggestion we give to use is *two* ways
to install Python packages (and 2 or 3 ways to virtualize a Python instance).
Note that one of my requirements was that "pip install foo" *must* do
the right thing in conda environments (whatever we decide the "right
thing" means in that context). It was buried at the end of a long
email though, so it would have been easy to miss.

That means the instructions to new users can be simple and consistent
- use pip commands to manage Python things, conda commands to manage
other stuff. They'll likely discover in fairly short order that the
conda commands also work for Python things, but it can be explained
that not all environments are conda environments, and hence pip works
in more situations than conda does, but at the cost of being specific
to Python packages.

Cheers,
Nick.
--
Nick Coghlan | ***@gmail.com | Brisbane, Australia
Donald Stufft
2014-04-18 21:22:36 UTC
Permalink
Post by Nick Coghlan
Post by Donald Stufft
So I’m not really worried about a competition or anything. I’m mostly worried
about confusion of users. What you’re suggestion we give to use is *two* ways
to install Python packages (and 2 or 3 ways to virtualize a Python instance).
Note that one of my requirements was that "pip install foo" *must* do
the right thing in conda environments (whatever we decide the "right
thing" means in that context). It was buried at the end of a long
email though, so it would have been easy to miss.
That means the instructions to new users can be simple and consistent
- use pip commands to manage Python things, conda commands to manage
other stuff. They'll likely discover in fairly short order that the
conda commands also work for Python things, but it can be explained
that not all environments are conda environments, and hence pip works
in more situations than conda does, but at the cost of being specific
to Python packages.
Cheers,
Nick.
--
Thinking about that and the implications of it.

Next question, where even is the code for “Anaconda”? I tried to download it
from their website and it’s behind an email form, I saw a link for github issues
but it’s in a dedicated “anaconda-issues” repo which doesn’t have any code
associated with it.

Also to be honest i’m a little uncomfortable with the idea of Python.org pushing
a platform where the company that develops the platform sells Add-ons to that
platform. So while Anaconda itself may be free and open source the fact that
the Anaconda distribution is a gateway to a particular company’s paid add ons
makes me feel a bit like a government sponsored monopoly kind of thing? I’m
not using good words here to describe what I mean, but it feels kind of icky to me.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Nick Coghlan
2014-04-18 21:40:46 UTC
Permalink
Post by Donald Stufft
Post by Nick Coghlan
So I’m not really worried about a competition or anything. I’m mostly worried
about confusion of users. What you’re suggestion we give to use is *two* ways
to install Python packages (and 2 or 3 ways to virtualize a Python instance).
Note that one of my requirements was that "pip install foo" *must* do
the right thing in conda environments (whatever we decide the "right
thing" means in that context). It was buried at the end of a long
email though, so it would have been easy to miss.
That means the instructions to new users can be simple and consistent
- use pip commands to manage Python things, conda commands to manage
other stuff. They'll likely discover in fairly short order that the
conda commands also work for Python things, but it can be explained
that not all environments are conda environments, and hence pip works
in more situations than conda does, but at the cost of being specific
to Python packages.
Cheers,
Nick.
--
Thinking about that and the implications of it.
Next question, where even is the code for “Anaconda”? I tried to download it
from their website and it’s behind an email form, I saw a link for github issues
but it’s in a dedicated “anaconda-issues” repo which doesn’t have any code
associated with it.
Also to be honest i’m a little uncomfortable with the idea of Python.org pushing
a platform where the company that develops the platform sells Add-ons to that
platform. So while Anaconda itself may be free and open source the fact that
the Anaconda distribution is a gateway to a particular company’s paid add ons
makes me feel a bit like a government sponsored monopoly kind of thing? I’m
not using good words here to describe what I mean, but it feels kind of icky to me.
I actually agree and it wouldn't necessarily make sense for *Anaconda*
itself to be the promoted sumo distribution. I'm more interested in
the idea of making a sumo distribution the default recommended entry
point, and handling the issues of independent curation and
redistribution that would come with that, as well as the mechanics of
how to actually handle installation and updates from the end user
perspective.

Perhaps we can get the "pip install ipython" experience to a good
place faster than I currently expect, and we can duck this entire
question (at least for Windows and Mac OS X).

Regardless, I think we've drifted far enough into speculative
territory to be off-topic for python-dev. This isn't a question we're
going to resolve in a hurry, and the near term focus should likely be
on improving the numpy+wheel story, since that's valuable regardless
of what happens with the python.org download pages. While that is
ongoing, people can mull over the various possibilities independently.

Regards,
Nick.
--
Nick Coghlan | ***@gmail.com | Brisbane, Australia
Paul Moore
2014-04-18 21:48:09 UTC
Permalink
Post by Nick Coghlan
Perhaps we can get the "pip install ipython" experience to a good
place faster than I currently expect, and we can duck this entire
question (at least for Windows and Mac OS X).
Huh? Last time I tried, it was pretty trivial.

pip install pyzmq pyreadline tornado ipython

(I might have missed a couple, as I did this from memory, not by
looking at the website to check). There's nothing that needs a
compiler apart from pyzmq and that has wheels for Windows these days,
and builds out of the box with no dependencies in any case. The rest
is all pure Python.

I've not tried with 2.0, but I did try one of the betas and it was
really that simple. (Windows 7 64-bit, Python 3.4 IIRC). And the 1.x
versions were just as simple.

Paul
Donald Stufft
2014-04-18 21:57:56 UTC
Permalink
Post by Paul Moore
Post by Nick Coghlan
Perhaps we can get the "pip install ipython" experience to a good
place faster than I currently expect, and we can duck this entire
question (at least for Windows and Mac OS X).
Huh? Last time I tried, it was pretty trivial.
pip install pyzmq pyreadline tornado ipython
(I might have missed a couple, as I did this from memory, not by
looking at the website to check). There's nothing that needs a
compiler apart from pyzmq and that has wheels for Windows these days,
and builds out of the box with no dependencies in any case. The rest
is all pure Python.
I've not tried with 2.0, but I did try one of the betas and it was
really that simple. (Windows 7 64-bit, Python 3.4 IIRC). And the 1.x
versions were just as simple.
Paul
Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know what that
includes. I’ve never used ipython except for the console.

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Paul Moore
2014-04-18 22:17:27 UTC
Permalink
Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know what that
includes. I’ve never used ipython except for the console.
The hard bit is the QT Console, but that's because there aren't wheels
for PySide AFAICT.

Paul
Nick Coghlan
2014-04-18 22:24:38 UTC
Permalink
Post by Paul Moore
Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know what that
includes. I’ve never used ipython except for the console.
The hard bit is the QT Console, but that's because there aren't wheels
for PySide AFAICT.
IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
let you break programming out of the low level box of controlling the
computer, and connect it directly to the more universal high level
task of understanding and visualising the world.

Regards,
Nick.
Post by Paul Moore
Paul
--
Nick Coghlan | ***@gmail.com | Brisbane, Australia
Donald Stufft
2014-04-18 22:28:34 UTC
Permalink
Post by Nick Coghlan
Post by Paul Moore
Post by Donald Stufft
Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know what that
includes. I’ve never used ipython except for the console.
The hard bit is the QT Console, but that's because there aren't wheels
for PySide AFAICT.
IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
let you break programming out of the low level box of controlling the
computer, and connect it directly to the more universal high level
task of understanding and visualising the world.
Regards,
Nick.
Post by Paul Moore
Paul
--
FWIW It’s been David Cournapeau’s opinion (on Twitter at least) that some/all/most
(I’m not sure exactly which) of these can be handled by Wheels (they just aren’t right now!).

-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
Nick Coghlan
2014-04-18 22:37:07 UTC
Permalink
Post by Nick Coghlan
Post by Paul Moore
Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know what that
includes. I’ve never used ipython except for the console.
The hard bit is the QT Console, but that's because there aren't wheels
for PySide AFAICT.
IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
let you break programming out of the low level box of controlling the
computer, and connect it directly to the more universal high level
task of understanding and visualising the world.
Regards,
Nick.
Post by Paul Moore
Paul
--
FWIW It’s been David Cournapeau’s opinion (on Twitter at least) that some/all/most
(I’m not sure exactly which) of these can be handled by Wheels (they just aren’t right now!).
Yeah, I think they're fixable too. And after thinking through the
implications of recommending a specific sumo distribution, that
actually does seem to be a more straightforward path as a "default
entry point".

I still see merit in working with the conda folks to make it easier
for Windows and Mac OS folks to keep their Python installations up to
date, and for Linux users to stay out of the system Python in a distro
independent manner, but that's a separate discussion from the
python.org download pages one.

Cheers,
Nick.
--
Nick Coghlan | ***@gmail.com | Brisbane, Australia
Donald Stufft
2014-04-18 22:59:36 UTC
Permalink
Post by Nick Coghlan
Post by Donald Stufft
Post by Nick Coghlan
Post by Paul Moore
Post by Donald Stufft
Maybe Nick meant ``pip install ipython[all]`` but I don’t actually know what that
includes. I’ve never used ipython except for the console.
The hard bit is the QT Console, but that's because there aren't wheels
for PySide AFAICT.
IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
let you break programming out of the low level box of controlling the
computer, and connect it directly to the more universal high level
task of understanding and visualising the world.
Regards,
Nick.
Post by Paul Moore
Paul
--
FWIW It’s been David Cournapeau’s opinion (on Twitter at least) that some/all/most
(I’m not sure exactly which) of these can be handled by Wheels (they just aren’t right now!).
Yeah, I think they're fixable too. And after thinking through the
implications of recommending a specific sumo distribution, that
actually does seem to be a more straightforward path as a "default
entry point".
I still see merit in working with the conda folks to make it easier
for Windows and Mac OS folks to keep their Python installations up to
date, and for Linux users to stay out of the system Python in a distro
independent manner, but that's a separate discussion from the
python.org download pages one.
Sure, and for *nix based ones there’s also pyenv which I personally use
and like :)
Post by Nick Coghlan
Cheers,
Nick.
--
-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
David Cournapeau
2014-04-19 08:36:23 UTC
Permalink
Post by Donald Stufft
Post by Nick Coghlan
Post by Paul Moore
Maybe Nick meant ``pip install ipython[all]`` but I don’t actually
know what that
Post by Nick Coghlan
Post by Paul Moore
includes. I’ve never used ipython except for the console.
The hard bit is the QT Console, but that's because there aren't wheels
for PySide AFAICT.
IPython, matplotlib, scikit-learn, NumPy, nltk, etc. The things that
let you break programming out of the low level box of controlling the
computer, and connect it directly to the more universal high level
task of understanding and visualising the world.
Regards,
Nick.
Post by Paul Moore
Paul
--
FWIW It’s been David Cournapeau’s opinion (on Twitter at least) that
some/all/most
(I’m not sure exactly which) of these can be handled by Wheels (they just
aren’t right now!).
Indeed, and the scipy community has been working on making wheels for new
releases. The details of the format does not matter as much as having one
format: at Enthought, we have been using the egg format for years to deploy
python, C/C++ libraries and other assets, but we would have been using
wheels if it existed at that time. Adding features like pre remove/post
install to wheels would be great, but that's a relatively simpler
discussion.

I agree with your sentiment that the main value of sumo distributions like
anaconda, active python or our own canopy is the binary packaging + making
sure it all works together. There will always be some limitations in making
those sumo distributions work seamlessly with 'standard' python, but those
are pretty much the same issues as e.g. linux integrators have.

If the python packaging efforts help the linux distribution integration, it
is very likely to help us too (us == sumo distributions builders) too.

David
Paul Moore
2014-04-18 21:40:19 UTC
Permalink
Post by Nick Coghlan
Note that one of my requirements was that "pip install foo" *must* do
the right thing in conda environments (whatever we decide the "right
thing" means in that context).
Is this specifically a requirement for conda? Or do you expect the
same to be true for ActivePython, Python(X,Y) and whatever other
distributions exist? Assuming conda isn't a special case, is it down
to pip to ensure that or to the distributions to play nicely with pip?

It's starting to feel as if conda *is* being treated as somehow a
special case. (At least in the sense that you're advocating that we
promote a non-minimalist distribution on the python.org website, and
you've explicitly stated that you personally like conda). I'm strongly
against the official python.org website promoting *any* external
distribution as the recommended solution. I'm not against having a
simple list of "alternative distributions" if you think that'd help,
but that's a far cry from "advertising a full featured sumo
distribution that lets new users quickly experience the full power and
flexibility of things like IPython notebooks".

On the other hand, it might be good to have an explicit advertising
page - "See the power of Python" which *did* direct users to stacks
that showcase particularly impressive uses of Python. There could be a
section about the IPython notebook (which mentions that the conda
Python stack comes with IPython preinstalled) and another about Django
(hmm, what's the curated stack for web developers?) and another
showing the Visual Stiudio Tools for Python (hmm, stack for Windows
developers/admins?).

As a comparison, java.com points you straight at the standard Java SE
download. You don't see JBoss or Weblogic promoted on there, even
though those are likely to be how most developers interact with Java.
To get anything other than the minimal distribution, you need to know
you want it and search for it.

Paul
Paul Moore
2014-04-18 20:27:51 UTC
Permalink
Post by Nick Coghlan
At this point, however, I'm mainly looking for consensus that there
*are* two different problems to be solved here, and solving them both
well in a single tool is likely to be nigh on impossible. (I'm aware
we're really on the wrong list for that discussion, but I also think
there's value in getting some broader python-dev awareness of this
particular topic)
I'm not entirely sure what you're proposing here.

Obviously there are multiple classes of users (whether it's as simple
as just two, or whether there's more is less important).
Clearly there is a case for curated stacks and OS distributions, and
clearly some people use those stacks and distributions.
Things aren't perfect - CPython and projects like pip need to be aware
of, and respond to, the differing needs of people who use Python in
different ways.

But what are you suggesting python-dev needs to *do* about this? What
precisely is wrong with how we deal with the multiple types of user
that exist at the moment?

Without wanting to sound like I'm taking things personally, it feels
like there's an intention to suggest to *some* people that they would
be better served by a curated stack. I don't know how to answer that
except from a personal perspective[1], and it's hard to do that
without knowing whether I'm in a group that you'd see as benefiting
from a curated stack.

One thing I *would* suggest is that a lot of "corporate" use of Python
(by which I mean semi-informal scripting and automation done as part
of the infrastructure of larger projects written in more "enterprise"
tools like Java or higher level CRM/eBusiness/etc packages) is not
suitable for a curated stack (corporate IT policies would see that as
a "3rd party tool" where the python.org distribution is just a
project-internal utility). But the staff involved are typically not
familiar with open source or its culture, and struggle to deal with
things like package management (this is *not* the "legal approval"
issue, as cut and paste of code from websites is common in this
culture - it's "internal use only"). Within the context of your two
categories, this may well be a third one (unless you stretch
"application developers" way beyond what I think you are intending).

Paul

[1] By which I mean "looking at what I use Python for, how I see it
used by others in my organisation, and how I would expect to promote
Python to people who don't currently use it but whom I feel would
benefit from it".
Nick Coghlan
2014-04-18 20:59:35 UTC
Permalink
Post by Paul Moore
Post by Nick Coghlan
At this point, however, I'm mainly looking for consensus that there
*are* two different problems to be solved here, and solving them both
well in a single tool is likely to be nigh on impossible. (I'm aware
we're really on the wrong list for that discussion, but I also think
there's value in getting some broader python-dev awareness of this
particular topic)
I'm not entirely sure what you're proposing here.
Obviously there are multiple classes of users (whether it's as simple
as just two, or whether there's more is less important).
Clearly there is a case for curated stacks and OS distributions, and
clearly some people use those stacks and distributions.
Things aren't perfect - CPython and projects like pip need to be aware
of, and respond to, the differing needs of people who use Python in
different ways.
But what are you suggesting python-dev needs to *do* about this? What
precisely is wrong with how we deal with the multiple types of user
that exist at the moment?
When you go to python.org, you are currently advised to download one
of the following:

- a source tarball (Linux)
- a bare bones binary installer (Windows & Mac OS X)

What I am advocating for is that *we are currently doing it wrong*, as
these are unlikely to be the best thing to install for most new Python
users. Now, we could make things even *more* confusing by instead
presenting the prospective user with *even more* choices, like "oh,
you might want to get it from your Linux distro, or maybe homebrew, or
MacPorts", but that's not helping, because it's asking new users to
make choices that they may not be in a position to make.
Post by Paul Moore
From a usability perspective, I don't believe we should be advertising
a minimalist installation as the preferred entry point for new users -
we should be advertising a full featured sumo distribution that lets
new users quickly experience the full power and flexibility of things
like IPython notebooks. I like Anaconda, because it's fully open
source, abstracts away the underlying operating system more than the
standard installers and the same mechanism that you use to update
packages can also be used to update the Python interpreter itself.

Everything else we do today should remain available for those that
want them - I'm not advocating taking anything away. But we need to
start getting more opinionated on behalf of new users that don't yet
have the experience they need to start challenging our judgement and
make their own choices about which tools to use out of the vast array
of choices the Python ecosystem provides.
Post by Paul Moore
Without wanting to sound like I'm taking things personally, it feels
like there's an intention to suggest to *some* people that they would
be better served by a curated stack. I don't know how to answer that
except from a personal perspective[1], and it's hard to do that
without knowing whether I'm in a group that you'd see as benefiting
from a curated stack.
No, I don't think you'd benefit substantially from a curated stack - I
don't think any experienced Python user would. But think through your
own personal stack. Start from a Google search for "Python tutorial"
and see how easily you can figure out how and where to obtain all
those components.

Putting a sumo distribution front and centre on the website
drastically improves the out of the box experience for new users,
*without* introducing a huge maintainability issue for the standard
library. New users gain quick access to a much larger fraction of the
overall Python ecosystem, while on the back end software *production*
side of things, everything stays decoupled.

It also provides a clearer timeline for adoption of new versions of
Python - while the reference interpreter would go up immediately, the
sumo version would only be updated once all the contained projects had
been checked for compatibility with the new version.
Post by Paul Moore
One thing I *would* suggest is that a lot of "corporate" use of Python
(by which I mean semi-informal scripting and automation done as part
of the infrastructure of larger projects written in more "enterprise"
tools like Java or higher level CRM/eBusiness/etc packages) is not
suitable for a curated stack (corporate IT policies would see that as
a "3rd party tool" where the python.org distribution is just a
project-internal utility). But the staff involved are typically not
familiar with open source or its culture, and struggle to deal with
things like package management (this is *not* the "legal approval"
issue, as cut and paste of code from websites is common in this
culture - it's "internal use only"). Within the context of your two
categories, this may well be a third one (unless you stretch
"application developers" way beyond what I think you are intending).
No, that is the case covered by 'creators of corporate "Standard
Operating Environment" definitions'. That's explicitly in the software
integrator category - whether those users formally have the
responsibility of defining an SOE, that's what they're doing in
practice.

Cheers,
Nick.
Post by Paul Moore
Paul
[1] By which I mean "looking at what I use Python for, how I see it
used by others in my organisation, and how I would expect to promote
Python to people who don't currently use it but whom I feel would
benefit from it".
--
Nick Coghlan | ***@gmail.com | Brisbane, Australia
Paul Moore
2014-04-18 21:19:24 UTC
Permalink
Post by Nick Coghlan
What I am advocating for is that *we are currently doing it wrong*, as
these are unlikely to be the best thing to install for most new Python
users.
For Windows users at least, I disagree. I have directed a lot of
people to the python.org Windows installers, and they have got up and
running without any issues (ignoring website usability "find the right
download" questions). Most of the time, they need nothing more than
the interpreter and the stdlib to become productive.

The downside to directing them at (say) conda is that 99.99% of the
documentation on the internet is not directed at conda users. Most of
the time, that's fine, because it's all generic stuff, but it's a trap
waiting to happen. I can't tell people to use conda myself as I'm not
a conda user, so I can't help them if they hit problems.
Post by Nick Coghlan
No, I don't think you'd benefit substantially from a curated stack - I
don't think any experienced Python user would. But think through your
own personal stack. Start from a Google search for "Python tutorial"
and see how easily you can figure out how and where to obtain all
those components.
Core python, plus a standalone copy of virtualenv (and on 3.4+ I don't
even need that). Everything else is installed from PyPI as needed.
Agreed that starting from a Google search won't get that at the
moment, but that's what the packaging initiatives are trying to solve
- get the infrastructure there so that more and more people say the
same things, and so google searches become more accurate.

My vision of the ideal is:

1. Install Python from python.org
2. py -m venv foo
3. foo\Scripts\activate
4. pip install <anything>

(Note that item 4 relies on pervasive availability of wheels for
packages like numpy, scipy, lxml, pywin32, etc)

Oh, and everyone needs to switch to Windows so this works :-)

More seriously, I genuinely think that this is the ideal solution for
*all* Windows users who don't have the sort of specialised needs that
mean a curated stack is right for them, and they know about it from
word of mouth via colleagues in the same situation.

I have no idea, and no opinion, on how practical it is to achieve this
level of simplicity on Unix or OSX. But I do think it's the right goal
to aim for for any cases where using "the system Python" is the wrong
answer.
Post by Nick Coghlan
Post by Paul Moore
One thing I *would* suggest is that a lot of "corporate" use of Python
(by which I mean semi-informal scripting and automation done as part
of the infrastructure of larger projects written in more "enterprise"
tools like Java or higher level CRM/eBusiness/etc packages) is not
suitable for a curated stack (corporate IT policies would see that as
a "3rd party tool" where the python.org distribution is just a
project-internal utility). But the staff involved are typically not
familiar with open source or its culture, and struggle to deal with
things like package management (this is *not* the "legal approval"
issue, as cut and paste of code from websites is common in this
culture - it's "internal use only"). Within the context of your two
categories, this may well be a third one (unless you stretch
"application developers" way beyond what I think you are intending).
No, that is the case covered by 'creators of corporate "Standard
Operating Environment" definitions'. That's explicitly in the software
integrator category - whether those users formally have the
responsibility of defining an SOE, that's what they're doing in
practice.
OK. In which case that category is far broader than I anticipated, and
includes people who cannot even write a Windows batch file or use a
command line without instruction (I can name names!!! ;-)) If that's
your view, though, then all I can say is that I have no experience of
anyone who isn't in the "software integrator" category, so I'll take
your word that there is only one other category :-)

Paul
Greg Ewing
2014-04-18 22:16:50 UTC
Permalink
Post by Nick Coghlan
there are actually now *two* main ways of consuming
Really? We'd better do something about that. We don't want
anyone consuming Python -- we want some left over for the
rest of us!

(I'm making a serious point -- it's annoying when people use
the word "consume" as though it were a synonym for "use".
It's not.)

When-I-consume-a-word-said-Humpty-Dumpty-ly,
Greg
Nick Coghlan
2014-04-18 22:21:07 UTC
Permalink
Post by Greg Ewing
Post by Nick Coghlan
there are actually now *two* main ways of consuming
Really? We'd better do something about that. We don't want
anyone consuming Python -- we want some left over for the
rest of us!
(I'm making a serious point -- it's annoying when people use
the word "consume" as though it were a synonym for "use".
It's not.)
I don't consider "use" a synonym for "consume" in this case, as
consuming an upstream project covers redistribution as well. Now, I
could write "use or redistribute" everywhere, but "consume" is
shorter.

Cheers,
Nick.
--
Nick Coghlan | ***@gmail.com | Brisbane, Australia
Antoine Pitrou
2014-04-19 09:32:52 UTC
Permalink
On Fri, 18 Apr 2014 11:58:59 -0400
Post by Nick Coghlan
* Linux distributions and other operating system vendors
* Sumo redistributions (commercial or otherwise)
* "Python based environments" (PTVS, Enthought Canopy, wakari.io,
Python Anywhere, etc)
* Software-as-a-Service developers
* Device manufacturers
* PC OEMs
* creators of corporate "Standard Operating Environment" definitions
* System integrators (IBM, Boeing et al)
* Application developers (from simple CLI tools to OpenStack)
I don't understand this. Why are SaaS developpers or application
developpers (oh, why are they separate, by the way? :-)) "software
integrators" rather than "end users"? While Linux distributions, OS
vendors and the like will build their own Python from source, normal
developers will usually rely on an already existing packaging of Python
(an installer, or a distribution-provided package).
Post by Nick Coghlan
For end users, Python is likely consumed as *part of something else*.
Well, even for some developers. Consider someone writing a Web
application with Django (*): are they interested in Django because of
Python, or in Python because of Django? How can you reliably make the
difference?

(*) or a video game with Ren'Py

Regards

Antoine.

Loading...