The Python mapping supports two forms of code generation: dynamic and static.
On this page:
Dynamic Code Generation in Python
Using dynamic code generation, Slice files are "loaded" at run time and dynamically translated into Python code, which is immediately compiled and available for use by the application. This is accomplished using the
Ice.loadSlice function, as shown in the following example:
For this example, we assume that
Color.ice contains the following definitions:
The code imports module
M after the Slice file is loaded because module
M is not defined until the Slice definitions have been translated into Python.
Ice.loadSlice Options in Python
Ice.loadSlice function behaves like a Slice compiler in that it accepts command-line arguments for specifying preprocessor options and controlling code generation. The arguments must include at least one Slice file.
The function has the following Python definition:
The command-line arguments can be specified entirely in the first argument,
cmd, which must be a string. The optional second argument can be used to pass additional command-line arguments as a list; this is useful when the caller already has the arguments in list form. The function always returns
For example, the following calls to
Ice.loadSlice are functionally equivalent:
In addition to the standard compiler options,
Ice.loadSlice also supports the following command-line options:
Generate code for all Slice definitions, including those from included files.
Generate checksums for Slice definitions.
Locating Slice Files in Python
If your Slice files depend on Ice types, you can avoid hard-coding the path name of your Ice installation directory by calling the
This function attempts to locate the
slice subdirectory of your Ice installation using an algorithm that succeeds for the following scenarios:
- Installation of a binary Ice archive
- Installation of an Ice source distribution using
- Installation via a Windows installer
- Package installation on Linux (DEB/RPM)
- Execution inside a compiled Ice source distribution
slice subdirectory can be found,
getSliceDir returns its absolute path name, otherwise the function returns
Loading Multiple Slice Files in Python
You can specify as many Slice files as necessary in a single invocation of
Ice.loadSlice, as shown below:
Alternatively, you can call
Ice.loadSlice several times:
If a Slice file includes another file, the default behavior of
Ice.loadSlice generates Python code only for the named file. For example, suppose
Process.ice as follows:
If you call
Ice.loadSlice("-I. Syscall.ice"), Python code is not generated for the Slice definitions in
Process.ice or for any definitions that may be included by
Process.ice. If you also need code to be generated for included files, one solution is to load them individually in subsequent calls to
Ice.loadSlice. However, it is much simpler, not to mention more efficient, to use the
--all option instead:
When you specify
Ice.loadSlice generates Python code for all Slice definitions included directly or indirectly from the named Slice files.
There is no harm in loading a Slice file multiple times, aside from the additional overhead associated with code generation. For example, this situation could arise when you need to load multiple top-level Slice files that happen to include a common subset of nested files. Suppose that we need to load both
Kernel.ice, both of which include
Process.ice. The simplest way to load both files is with a single call to
Although this invocation causes the Ice extension to generate code twice for
Process.ice, the generated code is structured so that the interpreter ignores duplicate definitions. We could have avoided generating unnecessary code with the following sequence of steps:
In more complex cases, however, it can be difficult or impossible to completely avoid this situation, and the overhead of code generation is usually not significant enough to justify such an effort.
Static Code Generation in Python
You should be familiar with static code generation if you have used other Slice language mappings, such as C++ or Java. Using static code generation, the Slice compiler
slice2py generates Python code from your Slice definitions.
Compiler Output in Python
For each Slice file
slice2py generates Python code into a file named
Using the file name
X.py would create problems if
X.ice defined a module named
X, therefore the suffix
_ice is appended to the name of the generated file to prevent name collisions.
The default output directory is the current working directory, but a different directory can be specified using the
--output-dir option. You can further customize the location of generated files using metadata.
In addition to the generated file,
slice2py creates a Python package for each Slice module it encounters. A Python package is nothing more than a subdirectory that contains a file with a special name (
__init__.py). This file is executed automatically by Python when a program first imports the package. It is created by
slice2py and must not be edited manually. Inside the file is Python code to import the generated files that contain definitions in the Slice module of interest.
For example, the Slice files
Syscall.ice both define types in the Slice module
OS. First we present
And here is
Next, we translate these files using the Slice compiler:
If we list the contents of the output directory, we see the following entries:
OS is the Python package that
slice2py created for the Slice module
OS. Inside this directory is the special file
__init__.py that contains the following statements:
Now when a Python program executes
import OS, the two files
Syscall_ice.py are implicitly imported.
Subsequent invocations of
slice2py for Slice files that also contain definitions in the
OS module result in additional
import statements being added to
OS/__init__.py. Be aware, however, that
import statements may persist in
__init__.py files after a Slice file is renamed or becomes obsolete. This situation may manifest itself as a run-time error if the interpreter can no longer locate the generated file while attempting to import the package. It may also cause more subtle problems, if an obsolete generated file is still present and being loaded unintentionally. In general, it is advisable to remove the package directory and regenerate it whenever the set of Slice files changes.
A Python program may also import a generated file explicitly, using a statement such as
import Process_ice. Typically, however, it is more convenient to import the Python module once, rather than importing potentially several individual files that comprise the module, especially when you consider that the program must still import the module explicitly in order to make its definitions available. For example, it is much simpler to state
rather than the following alternative:
Customizing Compiler Output using Metadata in Python
As we showed in the previous section, the compiler by default generates all files and package directories into the specified output directory. Furthermore, the generated code assumes that all of the generated files are globally accessible via the Python search path. Continuing with our last example, the package initialization file
OS/__init__.py will contain the statements:
An application that wants to import these Slice definitions must therefore ensure that its search path includes the directory containing
Syscall_ice.py, and the
For an application with many Slice files, the output directory can quickly become cluttered with generated
*_ice.py files. If you intend to install the compiler output into a common system directory such as
site-packages, you may want more control over the location of the files. Note that you can't simply move the
*_ice.py files into a different subdirectory without also adding that directory to the search path or modifying every generated file that imports those files.
A simpler solution is to add a global metadata directive named
python:pkgdir to each Slice file that specifies the directory into which the compiler should place the corresponding
*_ice.py file. This directive also affects the
import statement that the compiler emits whenever the generated code is referenced. The compiler treats the specified directory as being relative to the output directory denoted by
--output-dir, or to the current working directory if
--output-dir is not defined.
Let's add the metadata directive to our sample Slice files, starting with
And here is
We used the same directive,
python:pkgdir:OS, in both Slice files. The compiler now generates the following output:
The package initialization file
OS/__init__.py imports the
*_ice.py files as shown below:
Our metadata example specifies the same directory as the top-level Slice module
OS, which means the
*_ice.py files are placed into the top-level Python package
OS. This is convenient from an organizational standpoint but you are not required to use the same name. We could specify an arbitrary name, such as
python:pkgdir:OS_gen, and the compiler would generate:
Using an alternate directory as shown above does not affect the package in which the definitions are placed at run time. For example, your application would still refer to
OS.Process as before. The
python:pkgdir metadata only affects the directory in which the
*_ice.py file is placed.
The metadata directive can optionally specify a nested subdirectory, such as
python:pkgdir:OS/gen. The compiler would generate
OS/gen/Process_ice.py into the output directory, and this file would be imported as
OS.gen.Process_ice. Your metadata directive must use forward slashes when specifying a nested subdirectory.
Include Files in Python
It's important to understand how
slice2py handles include files. In the absence of the
--all option, the compiler does not generate Python code for Slice definitions in included files. Rather, the compiler translates Slice
#include statements into Python
import statements as described below.
Include Files with
When the include file contains a
python:pkgdir metadata directive, the specified directory is translated into a package name. For example, the metadata directive
python:pkgdir:OS/gen becomes the package name
OS.gen. If the name of the include file is
Process.ice, then the compiler generates
Include Files without
python:pkgdir metadata is not present, the compiler determines the full path name of the include file and creates the shortest possible relative path name for the include file by iterating over each of the include directories (specified using the
-I option) and removing the leading directory from the include file if possible. For example, if the full path name of an include file is
/opt/App/slice/OS/Process.ice, and we specified the options
-I/opt/App/slice, then the shortest relative path name is
OS/Process.ice after removing
/opt/App/slice. Any remaining slashes are replaced with underscores, so the
import statement for
There is a potential problem here that must be addressed. The generated
import statement for our example above expects to find the file
OS_Process_ice.py somewhere in Python's search path. However,
slice2py uses a different default name,
Process_ice.py, when it compiles
Process.ice.. To resolve this issue, we must use the
--prefix option when compiling
--prefix option causes the compiler to prepend the specified prefix to the name of each generated file. When executed, the above command creates the desired file name:
It should be apparent by now that generating Python code for a complex Ice application requires a bit of planning. In particular, it is imperative that you be consistent in your use of
#include statements, include directories, and
--prefix options to ensure that the correct file names are used at all times.
Of course, these precautionary steps are only necessary when you are compiling Slice files individually. An alternative is to use the
--all option and generate Python code for all of your Slice definitions into one Python source file. If you do not have a suitable Slice file that includes all necessary Slice definitions, you could write a "master" Slice file specifically for this purpose.
Static Versus Dynamic Code Generation in Python
There are several issues to consider when evaluating your requirements for code generation.
Application Considerations for Code Generation in Python
The requirements of your application generally dictate whether you should use dynamic or static code generation. Dynamic code generation is convenient for a number of reasons:
- it avoids the intermediate compilation step required by static code generation
- it makes the application more compact because the application requires only the Slice files, not the assortment of files and directories produced by static code generation
- it reduces complexity, which is especially helpful during testing, or when writing short or transient programs.
Static code generation, on the other hand, is appropriate in many situations:
- when an application uses a large number of Slice definitions and the startup delay must be minimized
- when it is not feasible to deploy Slice files with the application
- when a number of applications share the same Slice files
- when Python code is required in order to utilize third-party Python tools.
Mixing Static and Dynamic Code Generation in Python
Using a combination of static and dynamic translation in an application can produce unexpected results. For example, consider a situation where a dynamically-translated Slice file includes another Slice file that was statically translated:
The Slice file
Session.ice is statically translated, as are all of the Slice files included with the Ice run time.
Assuming the above definitions are saved in
App.ice, let's execute a simple Python script:
The code looks reasonable, but running it produces the following error:
Normally, importing the Glacier2 module as we have done here would load all of the Python code generated for the Glacier2 Slice files. However, since
App.ice has already included a subset of the Glacier2 definitions, the Python interpreter ignores any subsequent requests to import the entire module, and therefore the
PermissionsVerifier type is not present.
One way to address this problem is to import the statically-translated modules first, prior to loading Slice files dynamically:
The disadvantage of this approach in a non-trivial application is that it breaks encapsulation, forcing one Python module to know what other modules are doing. For example, suppose we place our
PermissionsVerifier implementation in a module named
Now that the use of Glacier2 definitions is encapsulated in
verifier.py, we would like to remove references to Glacier2 from the main script:
Unfortunately, executing this script produces the same error as before. To fix it, we have to break the
verifier module's encapsulation and import the
Glacier2 module in the main script because we know that the
verifier module requires it:
Although breaking encapsulation in this way might offend our sense of good design, it is a relatively minor issue.
Another solution is to import the necessary submodules explicitly. We can safely remove the Glacier2 reference from our main script after rewriting
verifier.py as shown below:
Using the rules defined for static code generation, we can derive the name of the module containing the code generated for
PermissionsVerifier.ice and import it directly. We need a second
import statement to make the Glacier2 definitions accessible in this module.
slice2py Command-Line Options
The Slice-to-Python compiler,
slice2py, offers the following command-line options in addition to the standard options:
Generate code for all Slice definitions, including those from included files.
Generate checksums for Slice definitions.
PREFIXas the prefix for generated file names.
Generating Packages in Python
By default, the scope of a Slice definition determines the module of its mapped Python construct. There are times, however, when applications require greater control over the packaging of generated Python code. For example, consider the following Slice definitions:
Other language mappings can use these Slice definitions as shown, but they present a problem for the Python mapping: the top-level Slice module
sys conflicts with Python's predefined module sys. A Python application executing the statement
import sys would import whichever module the interpreter happens to locate first in its search path.
A workaround for this problem is to modify the Slice definitions so that the top-level module no longer conflicts with a predefined Python module, but that may not be feasible in certain situations. For example, the application may already be deployed using other language mappings, in which case the impact of modifying the Slice definitions could represent an unacceptable expense.
The Python mapping could have addressed this issue by considering the names of predefined modules to be reserved, in which case the Slice module
sys would be mapped to the Python module
_sys. However, the likelihood of a name conflict is relatively low to justify such a solution, therefore the mapping supports a different approach: metadata can be used to enclose generated code in a Python package. Our modified Slice definitions demonstrate this feature:
The metadata directive
python:package:zeroc causes the mapping to generate all of the code resulting from definitions in module
sys into the Python package
zeroc. The net effect is the same as if we had enclosed our Slice definitions in the module
zeroc: the Slice module
sys is mapped to the Python module
zeroc.sys. However, by using metadata we have not affected the semantics of the Slice definitions, nor have we affected other language mappings.
python:package directive can also be applied as global metadata, in which case it serves as the default directive unless overridden by module metadata.