Ada-Python Demo

This article is a continuation (or rather completion) of the earlier article Ada-Python Binding, which presents the very basics of extending the Python interpreter with modules implemented in Ada.

This article shows the other side of the story - that is, extending the Ada program with plugins or scripts written in Python. In this case the Ada program has to include (it is called embedding here) a real Python interpreter, which will be asked to execute dynamically provided scripts.

These two approaches have different motivations:

• In the first case you want to extend the Python interpreter with modules written in Ada. This makes sense if you have a set of utilities written in Ada and those utilities do not form a complete program, but can be helpful in writing one. For example, a set of image-processing subprograms that are efficiently implemented and natively compiled can be used in a Python program to improve its performance. (Of course performance is not the only valid reason to reuse existing code.) In any case, you want to write your main program in Python and use Ada utilities in that program.
• In the latter approach you want to implement your main program in Ada, but you want to add to it some advanced configurability in terms of plugins that are loaded at run-time. Or, perhaps, you want your program to be scriptable, so that users can provide their own new commands or new behaviours that are composed of existing functional building blocks. The Emacs text editor is an example of program that is scriptable with Lisp commands, and many other examples exist with probably the most popular one found in web browsers that execute client-side scripts that are part of many web pages.

Even though both problems relate to the integration between two languages, there are some very important differences in what has to be done. Namely, in the case of modules implemented in Ada, the set of Ada subprograms has to be exported and recognized by the main Python program - these subprograms are provided by the loadable module and are directly related to the module's functionality. That is, if it is an image-processing utility, the exported functions will have something to do with image processing and this is what the Python script will deal with. Things are different with embedded interpreters (our second case), as the language integration happens between the main program and the interpreter and this is more or less similar independently on what is the actual script doing. That is, the functions that the Ada program has to access are related to Python's object model, package importing, type conversion, etc. - and this will be similar in all applications that embed Python interpreters. This means that it makes sense to think about a general-purpose package for embedding Python interpreters in Ada applications and use this single package independently on the actual problem domain.

This article is accompanied by a demo program (download it from here) that contains such general-purpose package with the following specification skeleton:

package Python is

   Interpreter_Error : exception;
   
   type Module is private;

   procedure Initialize (Program_Name : in String := "");
   procedure Finalize;
    
   procedure Execute_String (Script : in String);

   function Import_File (File_Name : in String) return Module;
   procedure Close_Module (M : in Module);
   
   --  Overloads for "all" needed combinations of parameters and return types:
   
   procedure Call (M : in Module; Function_Name : in String);
   function  Call (M : in Module; Function_Name : in String; A : in Integer; B : Integer) return Integer;
   --  ...
   
private
   --  not shown
end Python;

The simple package above allows to embed the Python interpreter in a general way, no matter what the actual scripts are supposed to do. The subprograms should be easy to understand:

• Initialize - initializes the Python interpreter, should be called before doing anything related to Python in the Ada program.
• Finalize - cleans up the interpreter's resources.
• Execute_String - executes arbitrary Python command provided as String (note that this single string can contain arbitrary number of Python constructs, not necessarily a single one-line command).
• Import_File - loads the Python script from the named file and imports the module implemented by this script. Note that the module "handle" is returned.
• Close_Module - closes the given module.

The most interesting are, however, the Call subprograms as they encapsulate the machinery that is necessary to invoke the Python function by name. In the simplest case the invocation looks like this (assuming the module was already loaded from Python file):

   Python.Call (M, "do_something");

The above line is enough to call do_something Python function that does not expect any parameters and that returns nothing.

The demo program refers also to several Python functions that take two integers and return an integer (they perform four basic calculations on the given arguments) and their use is equally simple:

   Result := Python.Call (M, "add", A, B);

where Result, A and B are Integer variables in Ada.

This approach aims at very close integration between Ada and Python layers, so that the application code does not have to deal with dynamic discovery of types or data structures, but has the following drawback: the set of Call subprograms has to be extended to cover all intended function signatures (that is, all expected combinations of parameter and return types). It might look like a massive work, but in fact it is not - the set of such subprograms will rather quickly converge to what is actually needed in the particular application domain.

Several tips can be of use to those programmers who will attempt to extend the example Python package:

• If the Python function expects parameters, the Ada layer has to prepare the tuple with appropriate size. There is a convenient Py_BuildValue function in the Python API that can do it, see the existing Call body for analogies and the Python docs for all possible format specifiers that this function can use.
• If the Python function returns something, it can be simply read if it is a single value, or it is a tuple that can be extracted with similar Py_ParseTuple function from the Python API. Classes can be handled via PyObject pointers in the way that is analogous to how modules are already used in this demo.
• Do not attempt to load Python modules that have very complex or dynamically changing interfaces with the intent to cleanly reflect such interfaces at the Ada level - it will be very cumbersome and it is not worth the effort. Create a simple Python wrapper for the target module and access that wrapper instead. That is, keep things as simple as possible at the border between these two languages.
• Be careful with reference counting of Python objects to avoid memory leaks and crashes.

The demo program (download it from here) that accompanies this article has two parts, in separate directories:

• ada-python - shows an Ada program that loads and uses functions from external Python script.
• python-ada - shows a Python program that loads and uses functions from the module implemented in Ada.

Follow the instructions given in README files in each part.

The demo program was prepared with Python 2.x in mind. Unfortunately, the Python 3.x branch is not compatible with 2.x at the level of interpreter API and porting the demo program to Python 3.x would require some tweaks. Another porting issue can be related to the particular location where the Python library files are installed - on a Linux system with Python 2.7 installed, the library file in question is libpython2.7.so and usually it is installed where the GNAT project builder can find it. Otherwise (and on other systems, for example on Windows) the GNAT linker options will have to be changed appropriately.

In any case - feel free to contact us if you would like to use the principles from this demo program and you need help.