Creating a (cross-)compiler

From Freepascal Amiga wiki
Jump to: navigation, search

This page is about creating a Free Pascal compiler that is able to cross-compile pascal source-code into an AROS compatible executable.

When first entering the realm of compiling the compiler and producing (cross-)compilers, things seems to get complicated pretty fast. One of the reasons for that is the fact that the Free Pascal compiler itself supports many targets (both platform and processor) on and for which you could (cross-)compile your pascal sources. Nowadays even Windows is not targeted to solely run on i386-processors alone anymore (just like AROS/Linux), which makes the whole process of building and setting up a (cross-)compiler a pretty intimidating process (even when that process is semi-automated using makefiles).

This wiki tries to guide you to the process of creating and setting up the Free Pascal compiler for your favorite platform and that is capable of producing AROS compatible executable files.

In order to be able to do so, this wiki describes the use of ALB's branch of the Free Pascal compiler, because without it Free Pascal would have no knowledge at all about AROS being a (valid) target. Eventually the aim is to get the AROS branch integrated into the main stream trunk of Free Pascal, but at the moment this is a long-term goal.

During tests and experiments performed to be able to write this wiki, some experience was obtained and most of that acquired knowledge is written down in this wiki. As such, this wiki does not claim to be an expert guide nor does it implies that everything you read in this wiki is correct. Please feel free to offer or make corrections and/or otherwise make suggestions concerning this wiki.

Preface

I've chosen to handle each OS target individually and doing it in such a way that is perhaps not the most efficient (handling each target from scratch), but hopefully explains why each step was performed and show the repetitive nature of those steps being taken. In that regards, this wiki tries to show that no matter which platform, you should be able to repeat the (same) steps for your preferred OS/target.

Do keep in mind though that currently the only possible target for AROS is the i386 processor. Preliminary work was done by ALB to support the x64 processor as well, but unfortunately the AROS 64-bit distribution is currently not in too perfect shape.

Although also explained in each individual sub-chapters, there are some requirements that have to be met before it is possible to be able to perform the described steps. The following list is a quick overview of those requirements:

  • Knowledge on how to open a shell and being able to issues commands from the command-line/shell as well as perform tasks like creating a directory, copying and/or moving files and being able to extract files from archives.
  • An already installed (and working) Free Pascal compiler for your platform (also referred to as host OS/platform).
  • The source-code of ALB's Free Pascal compiler that includes AROS specifics
  • In order to be able to cross-compile for AROS, the AROS Binutils for your host platform are required.
  • Something else I might have forgotten to mention here.

Note

At the time of writing, the sources of ALB are used that are dated 20.11.2013 and the provided archive has an md5 checksum of 664525131660bc796b271740cb436db6. Some of the instruction written down in this wiki only apply to this specific release of the sources (where possible this will be noted).

The Free Pascal build-system

Free Pascal uses a build-system that is similar to those used by other programming languages like the gcc compiler, which uses command-line tools to accomplish the (instructed) task.

Free Pascal (ships with and) uses basic Binutils, like the make command, to be able to build the compiler. In order to accomplish this task, the make tool uses so-called makefiles which gives the tool instructions on how certain situations should be handled and which (other) command(s) needs to be invoked.

In Free Pascal those makefiles are automatically created by the fpcmake command by using makefile.fpc files. The makefile.fpc files are distributed together with the rest of the Free Pascal source-code. If your platform is supported by Free Pascal then you should not have to worry about all these files (only when things go wrong, as it might be caused by an issue in one of the used makefiles).

Actually, that is also almost everything there is to say about the Free Pascal build-system. Of course, a lot of (difficult and important) technical stuff is going on behind the scenes when a build takes place but, the only part that for us that is really interesting to know/understand is how we can invoke the build-system to do what we want.

If you are interested to learn more about the build-system that is used by Free Pascal then you should definitely checkout Marco's Build FAQ. At least it was a tremendous help for writing this wiki (even though I (initially) did not understood even half of what was written in that FAQ).

The Binutils (and cross-compilation)

Before we can even begin building the Free Pascal compiler, we should first pay attention to something called GNU Binutils (or Binutils for short).

Although the article in that link sums things up nicely, it doesn't actually tell something about the importance or tell which precise Binutils would be required for the process of (cross-)compiling the Free Pascal compiler (as well as its usage in the process of cross-compiling with the (created) compiler).

Let's start with something which require less knowledge. Because the pages in this wiki you are reading right now expect you to have an already installed and working Free Pascal compiler for your host OS, the necessary Binutils for your host OS are already present (Free Pascal ships with them, and can be found in the bin directory of your Free Pascal installation).

What that means is that, the provided Binutils are capable (and responsible) for creating executable files for (at least) your host OS. Would you like to target other platforms, then the correct Binutils are required that support the target OS.

So therefore it is important to realize that the Binutils are responsible for producing the correct executable files. Which Binutils should be used depend on the chosen target. This also means you can have (and in case of AROS cross-compilation even require) several Binutils installed/configured on your host OS.

The following sub chapters tries to aid in obtaining and configuring the (required) AROS Binutils.


More information on cross-compilation and the Binutils:

The AROS Binutils

As previously explained, we need the AROS specific Binutils and we need them for the platform on which we want to develop.

Pre-compiled versions of the Binutils for several host platforms can be found in the AROS Archives, in particular in the Development – Cross category.

You can download the pre-compiled Binutils for your specific host OS. In case your host OS isn't listed there, that would mean you have to manually compile (or setup) the Binutils (which is usually the case for an host OS like Linux).

More details for several host platforms are being described in upcoming sub chapters.

Another important (and out of the ordinary) tool that is needed to let the AROS Binutils work correctly is collect-aros. This tool is AROS specific and needs to be integrated into the Binutils toolchain. More information on this (AROS specific) tool can be read in this thread.

The AROS Binutils and AROS

The easiest way to obtain the AROS Binutils would be to install the development package when you install Icaros Desktop (not available in the light version) or, in case you happen to use AspireOS, they are already installed by default.

Both mentioned distributions take care of setting up the AROS Binutils automatically and, no further steps are needed.

You can check if the Binutils are installed correctly, by opening up a shell and type which ld. In case the which command returns nothing, then the AROS Binutils are not installed, in case it reports back something, you'll notice the location in which the ld command is located.

AFAIK, the only other way to obtain the binaries of the AROS Binutils is by downloading a nightly AROS boot iso, extract the downloaded archive and copy the Binutils out of the .iso that was extracted. The Binutils can be found in the directory /Development/i386-aros/bin/

There is a tool (written by Magorium) for AROS available (ExtractBinutils) that let's you extract the Binutils automatically from an ISO file.

When the Binutils executable files are extracted, you have to make sure they are copied somewhere on your path (or add an extra path that is searched), so that AROS is able to find the files automatically for you.

Note: the above instructions are applicable for AROS-i386 only.

The AROS Binutils and Linux

The easiest way for a Linux host to 'obtain' the AROS Binutils, is to download the SDK from the AROS nightly download page (look for pc-i386-sdk), extract the archive and run the script named AROS-SDK-Install.

The script will guide you through the process of installing and setting up the AROS Binutils automatically.

More detailed information is written by ALB on this wikipage.

Be sure to add (export) the path to the freshly installed Binutils so that the Linux environment is capable of locating the files automatically.

The AROS Binutils and Windows

The compiled AROS Binutils for Windows can be obtained from the AROS archives.

Because having (yet other) dependencies complicate things further, the choice was made to use the MinGW Binutils (as the cygwin version requires you to have cygwin installed as opposite to the MinGW version). The collect-aros tool is distributed with the GCC compiler archive named i386-aros-gcc-4.5.2-migw32-bin.zip (yes, there is indeed a typo in there).

In order to setup the Binutils correctly the following steps should be taken

  • Download the latest MinGW binutils archive from the AROS archives. Place the archive in the Q:\Temp directory if you want to follow the rest of the instructions literally.
  • Download the latest MinGW GCC compiler from the AROS archives. Place the archive in the Q:\Temp directory if you want to follow the rest of the instructions literally.
  • Make sure you have opened a command line prompt and that the shell is reside in directory Q:\Temp
  • Make a new directory named Q:\Temp\binutils
Q:\Temp> mkdir Q:\Temp\binutils
  • Extract the bin directory from the archive i386-aros-binutils-2.19-1-mingw32-bin.zip into Q:\Temp\binutils (but don't create the bin directory in there. We simply want to place the executable files in the root of the Q:\Temp\binutils directory)
Using the unzip command line tool (part of fpc) that looks like:
Q:\Temp> unzip -j Q:\Temp\i386-aros-binutils-2.19-1-mingw32-bin.zip bin/*.* -d Q:\Temp\binutils
  • Extract the file collect-aros.exe in directory \i386-aros\bin from archive i386-aros-gcc-4.5.2-migw32-bin.zip to Q:\Temp\binutils.
Q:\Temp> unzip -j Q:\Temp\i386-aros-gcc-4.5.2-migw32-bin.zip i386-aros/bin/collect-aros.exe -d Q:\Temp\binutils
  • Rename the collect-aros.exe to i386-aros-collect-aros.exe
Q:\Temp> rename Q:\Temp\binutils\collect-aros.exe i386-aros-collect-aros.exe
  • Create environment variable COMPILER_PATH=Q:\Temp\binutils
Q:\Temp> set COMPILER_PATH=Q:\Temp\binutils


Please note that the setting of the environment variable (as shown) is just temporary. If you close the shell it will be removed. If you want to make the environment variable permanent (including surviving a reboot) then you would need to consult your OS manual to see how this should be done. For example microsoft explains it for windows XP here (if you happen to know the link to a KB article for another windows version, then please feel free to mentioned it so I can make a nice table with links for each individual Windows version).

The SDK factor

Besides the need of having the proper Binutils to be able to cross-compile successfully, there is another factor to consider, being the Software Development Kit (Or SDK for short) of/for the target platform.

The Free Pascal compiler is only able to produce meaningful executable files (for the chosen target) when it knows how to address certain functionality of that target system. This basic functionality is part of the so-called RTL and is platform dependent. Usually these dependencies are encapsulated in the (platform specific) SDK.

Besides the RTL, Free Pascal also supports 3th party extensions (such as SDL, OpenAudio, etc.) which are not part of the basic RTL but are distributed along FPC in separate packages. These packages rely heavily on the availability of the external libraries for the target platform (similar like the RTL).

In order to let the Free Pascal compiler make use of these 'libraries' it needs to link against those during a build-process. Therefore, Free Pascal needs to be able to locate these dependencies.

This can cause several issues, especially when targeting Linux OS for cross-compilation. Because every Linux distribution has its own (specific) set of dependencies, it is almost impossible to resolve those issues (without compiling those dependencies yourself) and because the linking process being used is static linking (in opposite of dynamic linking). More about the linking process (in general) can be read here.

Since most (if not all) of those dependencies (for Linux especially) are written in the c-programming language, it would be out of scope for Free Pascal to support this out of the box. Hence the reason it is advised to not cross-compile from another platform to Linux, but use Linux as your main development system and cross-compile for the other desired targets. See also this Free Pascal wiki section for more details (although specifically written for a windows host, it applies in generic, ergo also for AROS as being host).

This isn't a real issue for AROS and Windows being a target, as both do not require to link against static objects, rather they are able to 'use' these dependencies in a dynamic matter (e.g. they dynamically open up an external .dll or .library during run time, instead of being linked against during compile time). Ergo, if a .dll or .library file wouldn't exist (or could not be found), then an error would be returned during run time, but not during (cross-)compilation time.

Do note however, that with AROS it is theoretically possible to link against (in AROS SDK provided) static objects during compilation time. But the process of building the Free Pasal (cross-)compiler itself does not make any use of that feature.

Should you want to cross develop applications for AROS and want to link against these objects, then you are required to install the AROS SDK on your host OS as well as configure the Free Pascal compiler in such a way that it is able to find these linkable objects. Of course, if you would create your own (linkable AROS) object files, then you could do without the AROS SDK being installed.

Also note that due to the fact that the AROS linkable objects are currently heavily tight to the GCC-compiler and AROS build-system, that it is currently impractical to be able to link against objects from the AROS SDK.

Creating the compiler

Where things basically comes down to is that this wiki 'expects' to create the compiler for your host, and additionally create the compiler for AROS. That would require to follow instructions for at least two of the upcoming sub chapters.

That will result in two separate created archives that would need to be 'merged' together and which, together with the AROS Binutils, 'provides' a compiler that is a) able to run on your host OS and b) is able to cross-compile sources (on that same host OS) for AROS.

Of course, you do not have to follow this path and perhaps only want to follow those instructions that suits your specific situation/requirements better. Just keep in mind that this wiki does not provide any support for such.


Fixing things first

Unfortunately the currently provided archive (dated 20.11.2013) which contains the latest Free Pascal sources for AROS, contains sources that don't work out of the box (as expected).

For the technical people out there, the current Free Pascal is based on FPC trunk revision 18107 and as is with most other open source projects, things have been fixed past that revision (but the AROS branch wasn't kept in sync, due to several reasons).

Therefore, we would need to fix some things first. Don't worry, it is pretty simple to do and the wiki will provide the fixed files for you, so that you only would have to replace the existing 'faulty' files.

In that case, only follow and apply the sub-chapter “Fixing: Without manually having to edit files” and skip the rest of the 'fix' sub-chapters. The 'fix' sub-chapters are there for the technical inclined or for those that like to things manually (not to mention having the changes documented for historical reasons).

Fixing: Without manually having to edit files

Fix 1: aros-collect

Fix 2: fpcmake

Fix 3: fcl-web

Fix 4: removing existing fpmake

Creating the compiler - for AROS (on AROS)

Creating the compiler - for Linux (on AROS)

Creating the compiler - for Windows (on AROS)

Creating the compiler - for AROS (on Linux)

Creating the compiler - for Linux (on Linux)

Creating the compiler - for Windows (on Linux)

Creating the compiler - for AROS (on Windows)

Creating the compiler - for Linux (on Windows)

Creating the compiler - for Windows (on Windows)

Requirements

Step 1: Extracting the sources from the archive

Step 2: Fixing things

Step 3: A new fpcmake

Step 4: recreating the makefiles

Step 5: making the compiler

Step 6: making the AROS parts

Step 7: Putting it all together

Notes for the Windows platform

Putting things together