Difference between revisions of "XCDL packages"
m (→Repository content brief) |
m (→Outline of the build process) |
||
| Line 234: | Line 234: | ||
=== Outline of the build process === | === Outline of the build process === | ||
| − | + | The full build process is described in [[The build process]], but a summary is appropriate here. A build involves three directory structures: | |
| + | |||
| + | # The component repository. This is where all the package source code is held, along with CDL scripts, documentation, and so on. For build purposes a component repository is read-only. Application developers will only modify the component repository when installing or removing packages, via the administration tool. Component writers will typically work on just one package in the component repository. | ||
| + | # The build tree. Each configuration has its own build tree, which can be regenerated at any time using the configuration’s ecos.ecc savefile. The build tree contains only intermediate files, primarily object files. Once a build is complete the build tree contains no information that is useful for application development and can be wiped, although this would slow down any rebuilds following changes to the configuration. | ||
| + | # The install tree. This is populated during a build, and contains all the files relevant to application devel- opment. There will be a lib sub-directory which typically contains libtarget.a, a linker script, start-up code, and so on. There will also be an include sub-directory containing all the header files exported by the various packages. There will also be a include/pkgconf sub-directory containing various configura- tion header files with #define’s for the options. Typically the install tree is created within the build tree, but this is not a requirement. | ||
| + | |||
| + | The build process involves the following steps: | ||
| + | |||
| + | # Given a configuration, the component framework is responsible for creating all the directories in the build and install trees. If these trees already exist then the component framework is responsible for any clean-ups that may be necessary, for example if a package has been removed then all related files should be expunged from the build and install trees. The configuration header files will be generated at this time. Depending on the host environment, the component framework will also generate makefiles or some other way of building the various packages. Every time the configuration is modified this step needs to be repeated, to ensure that all option consequences take effect. Care is taken that this will not result in unnecessary rebuilds. | ||
| + | |||
| + | # The first step in an actual build is to make sure that the install tree contains all exported header files. All compilations will use the install tree’s include directory as one of the places to search for header files. | ||
| + | |||
| + | # All source files relevant to the current configuration get compiled. This involves a set of compiler flags initialized on a per-target basis, with each package being able to modify these flags, and with the ability for the user to override the flags as well. Care has to be taken here to avoid inappropriate target-dependencies in packages that are intended to be portable. The component framework has built-in knowledge of how to handle C, C++ and assembler source files — other languages may be added in future, as and when necessary. The compile property is used to list the files that should get compiled. All object files end up in the build tree. | ||
| + | |||
| + | # Once all the object files have been built they are collected into a library, typically libtarget.a, which can then be linked with application code. The library is generated in the install tree. | ||
| + | |||
| + | # The component framework provides support for custom build steps, using the make_object and make properties. The results of these custom build steps can either be object files that should end up in a library, or other files such as a linker script. It is possible to control the order in which these custom build steps take place, for example it is possible to run a particular build step before any of the compilations happen. | ||
| + | |||
| + | (TODO: The above steps are specific to eCos; XCDL will support both command line builds and Eclipse projects, with slightly different steps). | ||
=== Configurable source code === | === Configurable source code === | ||
Revision as of 19:35, 21 June 2014
For a package to be usable in the XCDL component framework it must conform to certain rules imposed by that framework. Packages must be distributed in a form that is understood by the component repository administration tool. There must be a top-level XCDL file which describes the package to the component framework. There are certain limitations related to how a package gets built, so that the package can still be used in a variety of host environments. In addition to these rules, the component framework provides a number of guidelines. Packages do not have to conform to the guidelines, but sticking to them can simplify certain operations.
Contents
Repositories
Packages and the local component repository
All development tools using XCDL packages include a local component repository. Similarly to the CMSIS Pack repository, this is a local folder structure where installed packages are located. The component framework comes with an administration tool that allows new packages or new versions of a package to be installed, old packages to be removed, and so on. Each package has its own little directory hierarchy within the component repository. Keeping several packages in a single directory is illegal. There are no strict rules defining where new packages should get installed.
To better accommodate the package separation for multi-vendor cases, the local folder hierarchy start with a folder with the vendor name.
Packages/ ├── ARM ├── ilg ├── Keil ├── Nuvoton ├── lwIP └── wolfSSL
Below each vendor there are hierarchies of packages. Unrelated packages are all stored just below the vendor folder.
Packages
├── ARM
│ └── CMSIS
├── Keil
│ ├── ARMCortex_DFP
│ ├── MDK-Middleware
│ ├── STM32F0xx_DFP
│ ├── STM32F1xx_DFP
│ ├── STM32F2xx_DFP
│ ├── STM32F3xx_DFP
│ ├── STM32F4xx_DFP
│ ├── STM32L0xx_DFP
│ ├── STM32L1xx_DFP
│ ├── STM32NUCLEO_BSP
│ ├── STM32W1xx_DFP
│ └── V2M-MPS2_CMx_BSP
├── lwIP
│ └── lwIP
└── wolfSSL
└── CyaSSL
Remote/archives repositories
To save space and to simplify management, each XCDL package is packed into a ZIP archive. To make these archives public, the usual method of distribution is via a web server, each archive having its own URL.
Individual packages
For individually distributed archived packages, the component framework should be able to manage these files, unpack and add their content to the local component repository.
Git/local development trees
For component developers the usual package life cycle of pack/publish/fetch/unpack is not only useless, but may have a significant impact on the speed of the debug/test cycle.
For these cases it should be possible to directly use local folders where the packages are already unpacked.
Since these development folders are usually linked to revision control systems (like Git), another useful feature for the component framework would be to directly manage remote Git repositories.
TODO: define details
Repository content brief
For each remote repository there is a summary content file enumerating the public packages with their full URLs and just enough information to build a brief outline of the package.
These files, usually named content.xml are managed by the administration tool. The various configuration tools read in these files when they start-up to obtain information about the various packages that have been installed.
For repositories of other types, like CMSIS Pack, which do not provide a content.xml file, the component framework provides a specific way of browsing the repository definition files and composing an equivalent content.xml, later cached locally.
An example of such file is presented below:
<?xml version="1.0" encoding="UTF-8"?>
<root version="1.1">
<repository name="Keil">
<description>Keil CMSIS packs repository</description>
<properties>
<property name="type">cmsis.repo</property>
<property name="repo.url">http://www.keil.com/pack/index.idx</property>
<property name="generator">GNU ARM Eclipse Plug-ins</property>
<property name="date">20140620141046</property>
</properties>
<packages>
<package name="CMSIS">
<description>CMSIS (Cortex Microcontroller Software Interface Standard)</description>
<versions>
<version name="4.1.0">
<description>- CMSIS-Driver 2.02 (incompatible update)</description>
<properties>
<property name="type">cmsis.pack</property>
<property name="vendor.name">ARM</property>
<property name="pack.name">CMSIS</property>
<property name="version.name">4.1.0</property>
<property name="archive.url">http://www.keil.com/pack/ARM.CMSIS.4.1.0.pack</property>
<property name="archive.name">ARM.CMSIS.4.1.0.pack</property>
<property name="archive.size">62260982</property>
<property name="dest.folder">ARM/CMSIS/4.1.0</property>
<property name="pdsc.name">ARM.CMSIS.pdsc</property>
<property name="date">2014-06-12</property>
</properties>
<outline>
<devicefamily name="ARM Cortex M0">
<property name="vendor.name">ARM</property>
<property name="vendor.id">82</property>
</devicefamily>
<devicefamily name="ARM Cortex M0 plus">
<property name="vendor.name">ARM</property>
<property name="vendor.id">82</property>
</devicefamily>
<devicefamily name="ARM Cortex M3">
<property name="vendor.name">ARM</property>
<property name="vendor.id">82</property>
</devicefamily>
<devicefamily name="ARM Cortex M4">
<property name="vendor.name">ARM</property>
<property name="vendor.id">82</property>
</devicefamily>
...
<board name="uVision Simulator">
<description>uVision Simulator</description>
<property name="vendor.name">Keil</property>
</board>
<component name="CMSIS / CORE">
<description>CMSIS-CORE for Cortex-M, SC000, and SC300</description>
</component>
<component name="Device / Startup">
<description>System and Startup for Generic ARM Cortex-M0 device</description>
</component>
<component name="Device / Startup / C Startup">
<description>System and Startup for Generic ARM Cortex-M0 device</description>
</component>
...
<example name="DSP_Lib Class Marks example (uVision Simulator)">
<description>DSP_Lib Class Marks example</description>
<property name="example.name">DSP_Lib Class Marks example</property>
</example>
<example name="DSP_Lib Convolution example (uVision Simulator)">
<description>DSP_Lib Convolution example</description>
<property name="example.name">DSP_Lib Convolution example</property>
</example>
...
</outline>
<external>
<board name="uVision Simulator">
<property name="vendor.name">Keil</property>
</board>
</external>
</version>
<version name="4.0.0">
<description>- CMSIS-Driver 2.00 Preliminary (incompatible update) ...</description>
<properties>
<property name="type">cmsis.pack</property>
<property name="vendor.name">ARM</property>
<property name="pack.name">CMSIS</property>
<property name="version.name">4.0.0</property>
<property name="archive.url">http://www.keil.com/pack/ARM.CMSIS.4.0.0.pack</property>
<property name="archive.name">ARM.CMSIS.4.0.0.pack</property>
<property name="archive.size">0</property>
<property name="dest.folder">ARM/CMSIS/4.0.0</property>
<property name="pdsc.name">ARM.CMSIS.pdsc</property>
</properties>
</version>
<version name="3.20.4">
<description>- CMSIS-RTOS 4.74 (see revision history for details) ...</description>
<properties>
<property name="type">cmsis.pack</property>
<property name="vendor.name">ARM</property>
<property name="pack.name">CMSIS</property>
<property name="version.name">3.20.4</property>
<property name="archive.url">http://www.keil.com/pack/ARM.CMSIS.3.20.4.pack</property>
<property name="archive.name">ARM.CMSIS.3.20.4.pack</property>
<property name="archive.size">52095025</property>
<property name="dest.folder">ARM/CMSIS/3.20.4</property>
<property name="pdsc.name">ARM.CMSIS.pdsc</property>
</properties>
</version>
...
</versions>
</package>
</packages>
</repository>
</root>
Package Versioning
Below each package directory there can be one or more version sub-directories, named after the versions. This is a requirement of the component framework: it must be possible for users to install multiple versions of a package and select which one to use for any given application. This has a number of advantages to users: most importantly it allows a single component repository to be shared between multiple users and multiple projects, as required; also it facilitates experiments, for example it is relatively easy to try out the latest version of some package and see if it makes any difference. There is a potential disadvantage in terms of disk space. However since XCDL packages generally consist of source code intended for small embedded systems, and given typical modern disk sizes, keeping a number of different versions of a package installed will usually be acceptable. The administration tool can be used to remove versions that are no longer required.
Packages/ilg
└── Xyzw
├── 3.20.3
├── 3.20.4
├── 4.1.0
└── current
The version current is special. Typically it corresponds to the very latest version of the package.
All other subdirectories of a package correspond to specific releases of that package. The component frame- work allows users to select the particular version of a package they want to use, but by default the most recent one will be used. This requires some rules for ordering version numbers, a difficult task because of the wide variety of ways in which versions can be identified.
Package contents and layout
A typical package contains the following:
- Some number of source files which will end up in a library. The application code will be linked with this library to produce an executable. Some source files may serve other purposes, for example to provide a linker script.
- Exported header files which define the interface provided by the package.
- On-line documentation, for example reference pages for each exported function.
- Some number of test cases, shipped in source format, allowing users to check that the package is working as expected on their particular hardware and in their specific configuration.
- One or more CDL scripts describing the package to the configuration system.
It is also conventional to have a per-package ChangeLog file used to keep track of changes to that package. This is especially valuable to end users of the package who may not have convenient access to the source code control system used to manage the master copy of the package, and hence cannot find out easily what has changed. Often it can be very useful to the main developers as well.
Any given packages need not contain all of these. It is compulsory to have at least one XCDL file describing the package, otherwise the component framework would be unable to process it. Some packages may not have any source code: it is possible to have a package that merely defines a common interface which can then be implemented by several other packages, especially in the context of device drivers; however it is still common to have some code in such packages to avoid replicating shareable code in all of the implementation packages. Similarly it is possible to have a package with no exported header files, just source code that implements an existing interface: for example an ethernet device driver might just implement a standard interface and not provide any additional functionality. Packages do not need to come with any on-line documentation, although this may affect how many people will want to use the package. Much the same applies to per-package test cases.
The component framework has a recommended per-package folder layout which splits the package contents on a functional basis:
Packages/ilg/Xyzw/current ├── ChangeLog ├── doc ├── include ├── meta │ └── xcdl.xml ├── src └── tests
For example, if a package has an include sub-folder then the component framework will assume that all header files in and below that folder are exported header files and will do the right thing at build time. Similarly if there is doc property indicating the location of on-line documentation then the component framework will first look in the doc sub-folder. This folder layout is just a guideline, it is not enforced by the component framework. For simple packages it often makes more sense to have all of the files in just one directory. For example a package could just contain the files hello.cpp, hello.h, hello.html and xcdl.xml. By default hello.h will be treated as an exported header file, although this can be overridden with the includeFiles property. Assuming there is a doc property referring to hello.html and there is no doc sub-directory then the tools will search for this file relative to the package’s top-level and everything will just work. Much the same applies to hello.cpp and xcdl.xml.
Outline of the build process
The full build process is described in The build process, but a summary is appropriate here. A build involves three directory structures:
- The component repository. This is where all the package source code is held, along with CDL scripts, documentation, and so on. For build purposes a component repository is read-only. Application developers will only modify the component repository when installing or removing packages, via the administration tool. Component writers will typically work on just one package in the component repository.
- The build tree. Each configuration has its own build tree, which can be regenerated at any time using the configuration’s ecos.ecc savefile. The build tree contains only intermediate files, primarily object files. Once a build is complete the build tree contains no information that is useful for application development and can be wiped, although this would slow down any rebuilds following changes to the configuration.
- The install tree. This is populated during a build, and contains all the files relevant to application devel- opment. There will be a lib sub-directory which typically contains libtarget.a, a linker script, start-up code, and so on. There will also be an include sub-directory containing all the header files exported by the various packages. There will also be a include/pkgconf sub-directory containing various configura- tion header files with #define’s for the options. Typically the install tree is created within the build tree, but this is not a requirement.
The build process involves the following steps:
- Given a configuration, the component framework is responsible for creating all the directories in the build and install trees. If these trees already exist then the component framework is responsible for any clean-ups that may be necessary, for example if a package has been removed then all related files should be expunged from the build and install trees. The configuration header files will be generated at this time. Depending on the host environment, the component framework will also generate makefiles or some other way of building the various packages. Every time the configuration is modified this step needs to be repeated, to ensure that all option consequences take effect. Care is taken that this will not result in unnecessary rebuilds.
- The first step in an actual build is to make sure that the install tree contains all exported header files. All compilations will use the install tree’s include directory as one of the places to search for header files.
- All source files relevant to the current configuration get compiled. This involves a set of compiler flags initialized on a per-target basis, with each package being able to modify these flags, and with the ability for the user to override the flags as well. Care has to be taken here to avoid inappropriate target-dependencies in packages that are intended to be portable. The component framework has built-in knowledge of how to handle C, C++ and assembler source files — other languages may be added in future, as and when necessary. The compile property is used to list the files that should get compiled. All object files end up in the build tree.
- Once all the object files have been built they are collected into a library, typically libtarget.a, which can then be linked with application code. The library is generated in the install tree.
- The component framework provides support for custom build steps, using the make_object and make properties. The results of these custom build steps can either be object files that should end up in a library, or other files such as a linker script. It is possible to control the order in which these custom build steps take place, for example it is possible to run a particular build step before any of the compilations happen.
(TODO: The above steps are specific to eCos; XCDL will support both command line builds and Eclipse projects, with slightly different steps).
Configurable source code
TBD
Exported header files
TBD
Package Documentation
TBD
Test Cases
TBD
Host-side support
TBD
Making a Package Distribution
TBD
The XCDL package distribution file format
TBD
Preparing XCDL packages for distribution
TBD
Credits
The content of this page is based on Chapter 2. Package Organizaion of The eCos Component Writer’s Guide, by Bart Veer and John Dallaway, published in 2001.
