bpo-30439: Add some helpful low-level functions for subinterpreters.

Doc/library/_interpreters.rst

@@ -0,0 +1,90 @@
+:mod:`_interpreters` --- Low-level interpreters API
+===================================================
+
+.. module:: _interpreters
+   :synopsis: Low-level interpreters API.
+
+.. versionadded:: 3,7
+
+--------------
+
+This module provides low-level primitives for working with multiple
+Python interpreters in the same runtime in the current process.
+
+More information about (sub)interpreters is found at
+:ref:`sub-interpreter-support`, including what data is shared between
+interpreters and what is unique.  Note particularly that interpreters
+aren't inherently threaded, even though they track and manage Python
+threads.  To run code in an interpreter in a different OS thread, call
+:func:`run_string` in a function that you run in a new Python thread.
+For example::
+
+   id = _interpreters.create()
+   def f():
+       _interpreters.run_string(id, 'print("in a thread")')
+
+   t = threading.Thread(target=f)
+   t.start()
+
+This module is optional.  It is provided by Python implementations which
+support multiple interpreters.
+
+It defines the following functions:
+
+.. function:: enumerate()
+
+   Return a list of the IDs of every existing interpreter.
+
+
+.. function:: get_current()
+
+   Return the ID of the currently running interpreter.
+
+
+.. function:: get_main()
+
+   Return the ID of the main interpreter.
+
+
+.. function:: is_running(id)
+
+   Return whether or not the identified interpreter is currently
+   running any code.
+
+
+.. function:: create()
+
+   Initialize a new Python interpreter and return its identifier.  The
+   interpreter will be created in the current thread and will remain
+   idle until something is run in it.
+
+
+.. function:: destroy(id)
+
+   Finalize and destroy the identified interpreter.
+
+
+.. function:: run_string(id, command)
+
+   A wrapper around :c:func:`PyRun_SimpleString` which runs the provided
+   Python program in the main thread of the identified interpreter.
+   Providing an invalid or unknown ID results in a RuntimeError,
+   likewise if the main interpreter or any other running interpreter
+   is used.
+
+   Any value returned from the code is thrown away, similar to what
+   threads do.  If the code results in an exception then that exception
+   is raised in the thread in which run_string() was called, similar to
+   how :func:`exec` works.  This aligns with how interpreters are not
+   inherently threaded.  Note that SystemExit (as raised by sys.exit())
+   is not treated any differently and will result in the process ending
+   if not caught explicitly.
+
+
+.. function:: run_string_unrestricted(id, command, ns=None)
+
+   Like :c:func:`run_string` but returns the dict in which the code
+   was executed.  It also supports providing a namespace that gets
+   merged into the execution namespace before execution.  Note that
+   this allows objects to leak between interpreters, which may not
+   be desirable.

Doc/library/concurrency.rst

@@ -29,3 +29,4 @@ The following are support modules for some of the above services:
    _thread.rst
    _dummy_thread.rst
    dummy_threading.rst
+   _interpreters.rst

Include/internal/pystate.h

@@ -80,6 +80,20 @@ PyAPI_FUNC(_PyInitError) _PyPathConfig_Calculate(
 PyAPI_FUNC(void) _PyPathConfig_Clear(_PyPathConfig *config);
 
 
+/* Cross-interpreter data sharing */
+
+struct _cid;
+
+typedef struct _cid {
+    void *data;
+    PyObject *(*new_object)(struct _cid *);
+    void (*free)(void *);
+
+    PyInterpreterState *interp;
+    PyObject *object;
+} _PyCrossInterpreterData;
+
+
 /* Full Python runtime state */
 
 typedef struct pyruntimestate {