The products that we are currently developing are becoming more and more complex. This complexity comes from the increasing demand for new software features to be delivered in less time. This is reaching a kind of absolute level beyond which we can’t no longer develop the software in a manageable, cost-effective way. To just keep adding more engineers to develop the software is no longer the solution. We are reaching the boundaries of what is possible.
By: Paul Zenden, systeem architect at Sioux in Eindhoven.
New products, new games, and new services, they are all getting more and more complex. Think of virtual/augmented reality games, wafer steppers, your smartphone, smart cars and internet of things in general. Why is that?
New products/services have to live up to our expectations. We are getting used to the current way of working and expect the next version of a product to do better, be faster, look nicer, have more pixels, be connected to the internet always and everywhere, etc. New features have to be added in less time to keep a competitive edge. Furthermore, we expect that applications can connect to and work with each other (e.g. use your smartphone to control the TV). This is all enforced by growth of the hardware capabilities.
However, there is a problem. In some domains, we are already reaching the point that the size of the software is becoming so big that it is almost impossible to maintain. It’s getting out of control. We can’t just keep adding more engineers to develop and maintain the software. F. Brooks already made this observation in 1995 in ‘The Mythical Man-Month’.
The solution lies in reducing the complexity and developing software in a smarter way. One technology I would like to highlight is functional programming. Functional programming is certainly not the only and single solution and is also not new, but it will help us to take a first step.
Functional programming concepts are already known and used since the 1950s, but they haven’t been applied very widely in the past decades. An important reason is that the available hardware lacked computing power and internal memory capacity what is needed to create industrial solutions. The object-oriented paradigm was more suited then. Now that the we have the availability of cheap and powerful multi-core, distributed systems and cheap memory, functional programming catches up.
Functional programming works with ‘immutable state’, functions with no side-effects and the fact that complex functions can be constructed from simple functions. Functions are treated in the mathematical interpretation of functions (so called ‘pure’ functions). Functions only transform the input data, producing output data, without changing the input data. When functions have no side effects, it becomes much easier to realise parallel execution and gain a performance increase by executing the application on a multi-core target.
Immutable state means that once a variable is assigned some value, it can never be changed. By making changes to the state impossible multi-threaded solutions can be created, without many of the common issues like race conditions (when accessing shared memory) or dead-locks.
It almost seems impossible to write useful programs without the ability to change the value of variables. But it is, by combining pure functions, immutable variables and recursion. I challenge you to find out for yourself.
Functional programs are less complex in the sense that they, generally speaking, achieve the same functionalities as object-oriented programs with less, and better quality code. (On Rosetta Code there are many examples of the functional programming language solutions that are much smaller than the object-oriented counterparts.) This leads to a lower risk of bugs, due to immutable state and functions with no side effects. Functional programming also provides a powerful way to create distributed systems, due to the easy realisation of parallel execution.
But, too bad, functional programming also introduces a new complexity for a lot of engineers. And that is that functional programming is a paradigm that is significantly different from object-oriented or procedural programming. To be able to develop efficient functional code, you first have to learn functional programming concepts and overcome the paradigm shift. And that is not so easy. This hinders a more wide-spread acceptance at this moment. But should this stop us from looking at the benefits of functional programming languages? No. Learning a new paradigm, language is done once, after that, you can
There are a lot of functional programming languages to choose from like Scala, F#, Erlang, Clojure and many more. It’s often possible to combine imperative languages with functional programming languages, and/or use a coding style in an imperative language that results in a functional way of solving the problem.
Did you know that the following popular products/applications all use Erlang: WhatsApp, RabbitMQ, SimpleDB (Amazon), CouchDB, eJabberd, GPRS/3G mobile network nodes (Ericsson)?
New, powerful frameworks are coming out that shield off difficult details and that make applying functional programming concepts more and more accessible. Examples are ELM for client side web applications and Phoenix, a server-side web framework. Another interesting new-comer is Elixir. Elixir is based on Erlang, a mature functional programming language and framework. Erlang is already in use in the telecom domain for more than 20 years. Elixir embeds the power of Erlang, but with a more accessible language derived to some extend from Ruby. Elixir also has impressive meta-programming features which allows the development of so called internal DSL’s. Another nice development is the use of Elixir in the embedded software domain.
Functional programming reduces code size, reduces the risk of bugs, enables easy parallelisation and thus increases the performance. It is one of the available means to help us manage the growing complexity in products and services. It is especially a technology to consider for concurrent, distributed applications.
For many software engineers functional programming is not a part of their skills, yet. And this makes applying functional programming on a broad scale difficult. We can all help to turn this around and start learning functional programming today!
Paul Zenden is an experienced system architect at Sioux in Eindhoven.
If you want to read some more:
There are many books about functional programming, and also numerous online courses available on sites like Coursera, EDX and Plural Sight. I enjoyed this course very much: Erik Meijers 'Funtional Programming Fundamentals'.
