cancel
Showing results for 
Search instead for 
Did you mean: 

Software on Silicon Blog

JamesT_21
Moderator
Moderator

One Build to Make Them All

(Note: Keith Wheeler wrote this, not me! To save him learning "yet another system" I posted for him. I'll make sure he sees any comments or discussion.)

Recently Jim Trudeau wrote about the power of GNU make as used in our ModusToolbox build system: It’s Make Time. One of my favorite things about it is that it is both cross-tool and cross-platform. What happens when I initiate a build in the Eclipse IDE is the same thing that happens from Visual Studio Code, and that’s the same thing that happens with a command line build – whether on a Windows, Mac, or Linux!

OneBuild.jpg

Let’s talk a little bit about how this happens. In any IDE/build system, somewhere (hidden or obvious) there is “something” that manages the build process: what files, where are they, compiler and linker options, and so on. Some IDEs do this in a proprietary way. Others use a makefile. We use GNU make, but we improved it.

For example, we decided to have our build system discover source files automatically for a project. In many IDEs you must explicitly add a source file to project. If you’ve ever manually configured a makefile you know you list the various source files that you want compiled. The ModusToolbox build system makes that task go away.

You don’t have to edit the ModusToolbox makefile when you add another .c file to your project. You don't need to use the IDE to add that to the build. It just happens, and it works. What if you’ve got say a library with some test code that you don't want built? Our build system will look for ".cyignore" files, and will skip any files or even full directories listed there. So what gets built, and how it gets built, is all managed in one place. What do I mean by that?

Some environments manage and update a makefile when you modify the project. When you click build a makefile is executed. Others, when you click build the IDE generates the makefile on-the-fly based on the state of the project at that moment, and then runs make on that file. This can lead to variation in what happens.

From the point of view of a developer who simply wants to build and debug code, which process happens isn’t really that important, particularly if that person is the sole developer on a project. But what happens if I have different developers working with various tools on different operating systems? What if the developers are working on Windows-based computers but the integration team has a Jenkins install on a Linux server?

In those cases, with the "behind the scenes" IDEs and different operating systems, there are different pieces of software between the source code and the hex file. Sure, the compiler may be the same, but there are links in the chain that are different, and the details of what happens and what’s produced can vary. The devil lives in the details. The differences may not be an issue, but serious devops folks don’t like those kinds of differences, and for good cause.

Wouldn't it be better if the build process on the developer's machine is identical to the one used in the Continuous Integration system? Enter ModusToolbox, and one build to make them all.

With the ModusToolbox build system, regardless of operating system, the build uses GNU make and gcc as the compiler. There is no difference between a build from the Eclipse IDE, Visual Studio Code, Windows command line, or Linux command line. The build system uses make, parses things like the .cyignore file, finds all of the source, compiles, and links. Done.

ModusToolbox does support multiple IDEs, and for IAR Embedded Workbench and Keil µVision, you can export the project into those tools and use them. How a team uses those tools to manage a build is, of course, up to the team. You get the advantages of those tools, but you may lose some of the consistency of the “one build” process I’m talking about here.

One of my favorite ways of using the "One Build" is to have automated builds linked to git pushes for continuous integration/deployment. Continuous deployment because our build system not only creates the final build artifact, it can also program a target board.

First an admission: I consider myself an embedded systems engineer. I’m most comfortable specifying microcontrollers, creating schematics, and writing C. I am not a devops person. While I’m very comfortable with git and Linux, I’m not an expert at either. I probably fall into the "know enough to be dangerous" category.

That said, I was really interested in learning more about this, so I decided to build my own continuous integration server. I dusted off an older 1U rack mount Linux system I had. Everyone has one of these in the garage or down in the basement, right?

I decided to use GitLab and Jenkins. Honestly one of the most complicated issues, for me at least, was getting these two software packages to play nicely on the same machine, since they are both accessed via the web. Setting my Jenkins to port 8080 and gitlab to port 8081 worked for me.

Configuring the Jenkins job was pretty straightforward. I connected the job to my git repository, and set it to "build when a change is pushed". I also set the job to delete the workspace for every build to be sure things are always up to date.

When I push a change to git, Jenkins cleans the workspace directory and clones the repository. Then Jenkins is ready to execute the commands necessary to get the necessary libraries, build the source code, and program the target board. All this happens with two commands in the ModusToolbox build system, make getlibs and make program.

At this point there are a couple of little tweaks to get everything to work. In Windows I’m starting my ModusShell via the Cygwin.bat in my ModusToolbox/tools_2.1/modus-shell directory. This gives me a shell that has environment variables already set to know the path to the ModusToolbox tools directory. With the Jenkins build, I had to tell the build system where I put the tools on my machine.

A neat Jenkins feature is that it saves the output from the build console for later review. So on the Jenkins build I want a verbose log. My two commands are simply:

make getlibs CY_TOOLS_DIR=/home/keith/ModusToolbox/tools_2.1/ VERBOSE=true

make program CY_TOOLS_DIR=/home/keith/ModusToolbox/tools_2.1/ VERBOSE=true

To be absolutely sure that I have the exact same build between my development system and my CI system I need only check the version of make used in both cases. There are no in between behind the scenes scripts or executables required, just standard GNU make and a ModusToolbox installation.

This “One Build to Make Them All” environment may not be so important to the individual developer. To a project manager and a devops team working with diverse team members and systems, this is golden. Everyone gets the same exact build, and there is one less big thing to worry about.

0 Likes
1 Comment
javiermelendezc
New Contributor

Hey @JamesT_21  

I’m basically at the same position as you regarding CI and git, but my project was built in PSOC Creator.

Is there any alternative to get a build in CI working on an online service as GitHub Actions?

Is there an equivalent of gnu make for these kind of projects?

Or do you have to stick with installing fully PSOC Creator and then use terminal to build them with cyprjctmngr? (That’s a long install for a build that will last 2 minutes). It gets even worse if you take into account that CI servers on some services are ephemeral and are not persisted on each push.

I’d thank any other suggestions 

0 Likes
About the Author
Been there, done that. Mostly. For software tools and developer support.