Hi, please introduce yourself.
Hi, my name is Bruce Tate, and I am an author and a serial entrepreneur. Right now, at Groxio I’m building coursework and we’re focusing on teaching the Elixir language.
I’ve written many books. When I have a consultancy or when I have a business, rather than trying to compromise on my product price, what I’ve always done when I have extra time - I write.
It’s something that I really like to do, and it’s something that has served me well in the past. I have, gosh, I’ve never really counted, maybe 10 to 15 books out, and then a few more with my name as an editor, and then a few more that have been published around the Groxio platform, so, yes, I write a lot.
Let’s start with an overview about where languages are. What are your thoughts about language, design, current trends, and learning?
I think it’s all related. We just came out of this period where the whole industry was focused on one language or a series of languages, primarily Java and the counterparts for Microsoft, the C# counterpart. I’ve always believed that in such periods, different types of innovations happen and different types of curiosity happen.
As we’ve rolled forward with the internet, that’s opened up standards and that’s also opened up the ability for the industry to support and to actually help people understand a variety of programming languages. There’s curiosity as other people see languages rolling into place.
I think as new programming languages have emerged, there’s been another emergence, and that’s people willing to use them in production environments, people willing to teach them, people wanting to write about them, and people exploring with them, so I think all of these things are related.
“I think as new programming languages have emerged, there’s been another emergence, and that’s people willing to use them in production environments, people willing to teach them, people wanting to write about them, and people exploring with them, so I think all of these things are related.”
What pushes the industry and individuals into creating, designing, and introducing new languages and new patterns? Where does the genesis of language start?
Yeah, that’s a great question. I strongly believe that languages start because they are solving an important problem. We could actually go back to the emergence of the Java programming language, and it solved a particular problem.
At the time, most of the programming languages were proprietary, and they were delivered on specific platforms. For example, you could see the programming languages emerge on Windows and Linux, the Linux Operating Systems, and proprietary operating systems like IBM’s OS/2 and others, but you could also see this need for the IT house, the glasshouse, if you will, to basically deploy these languages on a variety of different platforms. It was with Java, and Sun brilliantly solved the problem by allowing people to deploy the language in the browser space, right?
Java was a language that was created to solve a portability problem. As JSP and servlets were created, it also allowed the capability of building around the internet, and those open standards allowed them to deliver applications to any desktop. I believe that if you look at any of the major programming languages, they are solving particularly difficult problems.
“I believe that if you look at any of the major programming languages, they are solving particularly difficult problems.”
What are some of the problems that drive these design decisions?
I’ve spent the last 15 years or so working on programming language adoption, and one of the big adoptions that’s happening is the emergence of functional programming languages. Like other programming language families, the functional programming family solves an individual problem.
The problem is that computers are increasingly becoming multicore systems. What that means is that instead of building one particular architecture and laying that out on a silicon wafer, rather than building wafers with high resolutions with more and more components, we hit a wall. It turns out that we’re actually out of space on the piece of silicon because the insulators between components are three to seven atoms thick. You just can’t make them any smaller.
What we do instead is we take those wafers and we stack them up like plates in the pantry. When that happens, we have to build programming languages in a different way. Said another way, these languages have to walk and chew gum at the same time. They have to support concurrency, and so that’s what’s driving functional programming languages.
The reason that these languages tend to work on silicon that’s stacked, that silicon that has to work together. They work on a principle called immutability, which means instead of taking one value and changing that to another value, and then changing that to another value, functional programming languages create one value.
Say we’re building a counter, maybe it would create the integer one, and then we would send that through a function to increment it, and it would create a new integer two, and we would send that through a counter to increment it, and that would create a new integer three.
Well, it turns out it’s easier to share three things than to share one thing when all of the programming languages and all the dependencies are trying to change that one value, so concurrency is really the driver for functional programming languages.
“Concurrency is really the driver for functional programming languages.”
Functional languages are stateless. Could you explain what that means and how it affects developers when they’re trying to enter from having a background in a procedural language or an object-oriented language?
Most object-oriented languages are mutable. That means that when I interact with a programming language, say there’s a program that has a person. Maybe that person changes a profession, so what’s going to happen is we’re going to see a function that is going to access one of the attributes on that person and change it.
When those changes happen, that’s a very easy thing to understand for the object-oriented programmer when there’s one system working on it. When there are multiple systems working on it, you can imagine how complicated it would be to write a program of 10,000 processes that might also have to access the same person with values like profession changing underneath them.
What tends to happen is that we have to think of a whole new series of tools, and algorithms, and ways of organizing and designing code to deal with these mutable versus immutable problems. Since functional languages are based on mathematical functions, they kind of do that naturally, right?
For example, if I’m building a counter in a Java system, or a Python system, or a Ruby system, I’d build the counter, it would have a value, it would have a count in it, and then we would send messages to that object to increment the counter, and that value would continue to go up. In a functional language, what I do instead is think in terms of that plus function. I would have a total, and then I would send a function, then I would send that total through a function, maybe with another argument, one.
I could start with a value of one, and then send a plus function with another value of one, and it would add those together to make a two, and then I would send that two through another function, plus one, and I would handle my increments that way, where each function created a new value. Each operation makes a new value. That pattern of calling functions is called reduce. It’s how many functional systems work.
We had Lisp, and then Scheme, and then now there’s Elixir obviously, and Clojure, and Haskell, and Scala, and Idris? I don’t know how to pronounce it. How did this suddenly explode? Is it solely because we needed it because we were going for massive concurrency, because we were going for multiple-chip design? How did we get here?
Yeah, so that’s another good question. Programming languages always emerge out of need, and if that’s not always true, they’re definitely adopted out of need. The reason that we’re starting to see an increase is that there’s need, and some of what we’ve talked about is concurrency, but there are other concepts as well, right?
“Programming languages always emerge out of need, and if that’s not always true, they’re definitely adopted out of need.”
Lisp, a foundational functional programming language, was created in the late 1950s. Why was it important?
The Lisp programming language was created as a higher level of abstraction. As a way of thinking about programming in terms of math. But it wasn’t successful because the creators of the language weren’t able to bring users along with them.
There wasn’t a way to tie these mathematical concepts together to the practical concerns of what the language was doing. Said another way, Lisp was solving problems that the market wasn’t understanding yet. So Lisp became important later because we started recognizing that first mutability and immutability were important, and second, it became important for the computer to manage things that the programmers had to manage before.
A good example of this is there was a great book by O’Reilly called Hackers and Painters. And in this one, Paul Graham talks about building stores, or storefronts, virtual storefronts. The way that he did so was that rather than building storefronts, the team built a storefront generator.
This idea of templates and building things based on templates became much more important as languages evolved. And so sometimes language adoption works that way. We don’t see immediate adoption because, well, the needs of the community aren’t really focused on those areas yet. But sometimes those languages are so forward thinking that we come back and we pick up those ideas later, and that’s what happened with Lisp.
“Sometimes those languages are so forward thinking that we come back and we pick up those ideas later, and that’s what happened with Lisp.”
So what drove that adoption?
So initially nothing drove the adoption because we weren’t trying to solve those particular problems. We started to see small successes with Lisp because, well, people were trying to build.
Rather than having an army of 10 developers to build 10 virtual storefronts, they thought, “Hey, why don’t we have one developer building a storefront generator?” And that’s what they did. Lisp basically offered a series of capabilities that was very friendly for building these kinds of templates.
This template capability, the language feature, is called macros. A macro is a template for a program. And it turns out that macros are also important when you’re building a programming language. So Lisp became important. It rose to importance as the idea of this templating system became important.
Then it fell out of favor because Lisp got too fragmented. So it turns out that Lisp was so good at building programming languages that we built too many languages based on Lisp, but since it fragmented, it wasn’t a way to keep enough of the industry together to build a critical mass to keep any one community interested.
If you look at the last 5 to 10 years, understanding functional programming and using functional programming has become essential. It has exploded. How did we get from there to here?
Let me pick off one particular business problem that’s kind of near and dear to my heart. I’ve been following the Elixir language, and that basically sprung out of the work that Robert Virding, Joe Armstrong, and crew did at Erickson Labs. The beginning of that work was that the team needed to build something that was very reliable, and that team was looking for two different things.
First, the right set of abstractions to deal with mathematically complex concepts. They were building phone switches. And second, they were looking for the opportunity to build something that needed to be a bit more reliable than the systems of the past.
They settled on this system called Erlang. I believe that Erlang provided two, well, let’s call them three important innovations to what the market was doing. So we say that need basically drives language adoption. And in this case, one of the needs was to solve complex problems, and so there was a functional style of programming developed for that to happen.
So the second need was for something that was reliable, and the Erickson team built this thing called OTP. But the idea was really pretty basic.
Rather than trying to build a process that was reliable, the Erickson team split that problem into two pieces: a process to do some meaningful work, and then another process that was dedicated to observing the system, and doing something dead simple when anything went wrong. That’s just to turn it off and restart it. If you’ve ever seen that British comedy, The IT Crowd, OTP is, “Did you try turning it off and on again?” And so that was the second business problem. If something failed, Erlang would just let it crash.
“Rather than trying to build a process that was reliable, the Erickson team split that problem into two pieces: a process to do some meaningful work, and then another process that was dedicated to observing the system, and doing something dead simple when anything went wrong.”
The third business problem was something that could run embedded systems well. And in this case, an embedded system was running on a phone switch, and it meant more than just the portability. It meant that there needed to be ways to deal with things like communications protocols. There needed to be language constructs for taking things apart, like pattern matching to break the first 8 bytes and then 12 bytes and so on and so forth. But these communication protocols needed tools for them to evolve.
So Erlang evolved and did very well in its niche, but it effectively failed to capture the imagination of object-oriented developers because they weren’t experiencing the same types of needs until the last 15 or 20 years or so. In the last 15 or 20 years, we have had this problem we talked about earlier, where we’re running out of space on an individual wafer, so we have to stack wafers up and the programs that run on top need to be concurrent. Basically, the programming language has to match the architecture style, and also the problems that we’re solving are getting more and more complex. As we build more and more sophisticated applications, we need to be able to allow the languages to support those sophisticated problems.
The organization and design tools and the language need to be up to the job. I would say that Elixir was built based on the same early problems that Erlang was solving. We need to be more reliable, we need to be concurrent, we need to have better organizational tools. So José Valim, the creator of Elixir, also needed the tools like the ones in Lisp that allowed him to pick the language up by its bootstraps, to build Elixir in Elixir itself. And to do that, he needed macros. So he went out and built them.
So we have the development of Elixir from the point of view of the greater development community. What pushed this amazing adoption? Was it concurrency? Was it distribution? Was it writing at scale? Was it the need for that fault tolerance and fault monitoring? What made it so appealing and brought it users and adopters?
So there’s the mine mill: need drives adoption, need drives adoption, need drives adoption. That’s what it all is. And so you mentioned things like distribution and concurrency. Well, it pays to go back and look at where the industry was with Java and where the industry needed to move to make Java a viable solution. Or to make object-oriented programming a viable solution in certain types of companies.
“So there’s the mine mill: need drives adoption, need drives adoption, need drives adoption. That’s what it all is.”
One of the problems with Java over time was that it was a good language for building systems that were a little bit concurrent—where developers needed to have portability and support internet standards. But Java wasn’t necessarily a very productive environment.
One of the things that happened was people rebelled against the type system in Java, which was statically typed. It wasn’t really an ideal system for object-oriented languages and the rapid development people wanted to do. It didn’t really mesh well with the object-oriented way, with message passing and things like that.
So a couple of languages spilled out of the object-oriented family of languages, like Ruby and Python, and a few more like that, that were much more productive languages. They are beautiful languages, and I will always love them. But they actually made a couple of compromises to be more productive. One of the compromises was they were going to go in a much more dynamic direction, and also an interpreted direction instead of a compiled direction. That’s a tradeoff. Greater short-term productivity for less compiler support and potentially more bugs.
What happened is that the Ruby and the Python languages were starting to get a pretty large following for building something that was productive, and a little bit less reliable than what you could get out of Java. We were starting to see a growth of applications like Twitter that were originally built in something like Ruby, where it was very productive for them to build the initial application, but Ruby was much less able to solve high-scale and high-concurrency problems, and widespread distribution and things like that.
So we had these systems. For example, Twitter had this graphic that was famous. It was called the Fail Whale, and it would pop up when the system was under duress.
Sometimes those were programming problems, but more often than not, there were scalability problems. Many of the early Elixir adopters either came out of the Java community, where things weren’t very productive, or they came out of the Ruby community, where they were productive but wouldn’t scale as far as people needed them to.
Elixir provided a fusion that solved both of those problems in different ways. It solved the scalability problem based on the common ancestry with the Erlang language. And, it solved the productivity problem with the functional programming languages and a Ruby-like syntax that people already knew and loved.
“Elixir provided a fusion that solved both [scale and productivity] in different ways”
Could you tell me a little bit about the productivity improvement?
One of the things that happens when you’re dealing with an object-oriented language, or even a procedural language, is that you might have an object. Calling a function on that object wouldn’t necessarily tell you what the state or what the return of that particular function would be.
Object-oriented languages have something called side effects. Let’s say that I’m working with a counter. Well, if I call the increment message on a counter, then what I get will depend on the state of the counter. But if I call the plus function in a functional language with a one and a two, I’m always going to get a three. So having fewer side effects turns out to be a very big deal when I’m doing things like debugging.
I can remember when I was in the Ruby language, I started to work more and more with debuggers and things of that ilk so that I could start to trap bugs based on the particular state. I could catch the system as it was creating the conditions that were favorable for a particular bug to pop up.
Now, I haven’t had to do the same thing in Elixir for a good long while. And the reason was that in Elixir, if I had a set of inputs, it was always going to create the same output. Having this reliability, this immutability built into the system, made it a lot easier for me to think in terms of, “Well, when something breaks, I can get the input conditions and I could always reproduce the bug.” All I have to do to reproduce that problem is to throw the same set of inputs to the same function, and then voila, I’ve got the problem in front of me.
“[In a functional language like Elixir, all] I have to do to reproduce that problem is to throw the same set of inputs to the same function, and then voila, I’ve got the problem in front of me.”
In a purely functional language, how do you approach common tasks that people who are doing object-oriented programming would do? Things like creating unit tests, debugging, documentation, composition?
Yeah. So there’s a lot of “how do you” questions built into that one. Let me basically talk about the concept of what’s the same in a functional system, what’s simpler, what tasks go away, and maybe what’s more difficult.
Some of the things that become easier are, for example, reliability. So when I deploy a system in Elixir, I don’t have to build in some type of monitor that says, “Have you crashed?” because that is actually baked into the Elixir language itself.
By the same token, if I’m using another programming language that doesn’t have something like OTP, maybe I do need some type of big-brother process that watches resources or watches the health of the system. But there are certain things in a functional programming stack that are going to be different.
One of those is deployment. And deployment is just hard. There are a lot of different concerns based on the programming languages that you use when you go to deploy something. They’re not harder than typical object-oriented languages. They’re just different.
In typical object-oriented and even sometimes procedural language, these problems are solved. Maybe there isn’t an out-of-the-box solution for Elixir or Erlang or Closure. Or maybe the skills that you’d have to go get to solve those problems are a little bit different. So I think that the answer is that you basically have to learn a programming environment, and that’s part of the whole education system.
For example, when I worked with Ruby, I had a system that would manage documentation for me, and it was built based on the same build tool. Well, the same thing exists in the Elixir space. When I was dealing with Java, there was a testing philosophy and a testing framework that I would use. In fact, there were several of them, and in fact, the same thing is true in the Elixir space.
It turns out that in functional programming languages, there tend to be better solutions for testing because, well, it’s easier to imagine a testing scenario where the job of the testing framework is just to choose inputs and throw those inputs at a particular function.
If I can generate those inputs automatically and throw them at the function, then I can just look at the things that must be true. And that’s called property-based testing.
I would say that those particular things aren’t necessarily more difficult, they’re just different. When you’re talking about shifting from one language to another, that’s a lot to learn. It almost has to be.
There’s something that’s dramatically different, when you’re dealing with an object-oriented language, whether that’s statically typed or dynamically typed, and a functional language, whether that’s statically typed or dynamically typed. And that thing is design.
Typically on the internet, when you’re looking at documentation around a language, you can find the way that libraries work. You can find the way that overall systems work. It’s much harder to find information about idiomatic design and idiomatic abstractions.
For example, if I’m working with a typical request response web server, I know that what I’ve got to do is make a request, and based on that request, build a single response. Now, when I’m working with Phoenix LiveView or something similar, what I have to do is much different.
It’s a web program that somebody else wrote, where I actually make a request, and then that page has a life of its own. And the question is, how do I step into the life cycle of that page and understand exactly what’s going on under the hood?
That’s a much more difficult problem to solve for people new to Elixir, let alone for somebody who already knows functional programming languages well. It is different and much more difficult for someone who’s trying to learn object oriented-languages and shifting that knowledge to a functional programming language in the area of event sourcing, which this concept depends on.
You mentioned Phoenix and LiveView. Stepping into this world, how many tools, libraries, frameworks do you need to become familiar with to be effective at a production level?
Let’s agree that the library is a group of prepackaged functions that my program can call. A group of prepackaged functions that my library can call. Now this kind of framework is a program that someone else wrote that will call back into my code. In Elixir, and in other languages like Elixir, we use the term behavior, but it’s really just inversion of control. A behavior is effectively just a contract between the programmer and the framework. That’s just an agreed upon contract for when particular functions get called. It’s nothing but a plugin with a contract, really.
And you ask, how large is that set? Well, it’s pretty big. That’s unfortunate, but it’s reality. The problem is that the typical problems that we solve, they’ve gotten larger and more difficult and larger over time. And we need bigger libraries to solve them.
When I first started my career, I was a database specialist. All I needed to know was SQL, I needed to be able to wire that into my program, and then I would build my interface and then someone else would call that code. That’s not what’s true today.
Today we have a term called full-stack developer, and the definition of what a full-stack developer is changes with pretty good frequency so that it’s not enough for a person to know how the programming language works. In our case we’re talking about Elixir, but other languages have frameworks too. They also will need to know how databases work. There’s also an abstraction for how the database layer talks to the language layer, and there’s also an abstraction for how HTML or whatever user interface on rendering is created.
There’s also a framework for how that’s updated and sent back to the client. And if I’m also having to build JavaScript on the webpage, then suddenly every application that I’m building is a distributed application, and things get really, really demanding.
I would say a couple of things. The first is that it really helps to have a series or a collection of people with good foresight. I’ll mention José Valim, the creator of the language, and Chris McCord, the creator of Phoenix. One of the things that they do a good job of is insulating developers from having to know a lot of different dependencies. And that’s really, really valuable.
The second thing is it really helps building on abstractions that are similar all the way through the language. If we’re talking about the first argument in functions in the library, in Elixir, that first argument is typically going to be the type of things in that module.
I think that the pool of things that you have to know is absolutely massive. I think that as developers, we often try to think about and solve all of those problems ourselves. What we ought to be doing instead is enlisting help to actually make those breakthroughs and connections.
I’d like to pull you back just a little bit to Elixir, the BEAM, Ecto, LiveView, and so forth. Are there any other must-know language abstractions to be proficient in this environment?
There’s a family of applications that used to be request and response. So if you looked at early versions of Yahoo or of Facebook, the way that it worked is you typed a URL or you clicked on a link and your page would package up a request, send it across the internet and send back another page, and that was it. The user would look at those results. Maybe you would follow a link by clicking on a person that was on your Facebook feed.
Now, over time, those interfaces have become much more interactive. So what that means is that when you initially request that Facebook page, what you’re going to see is a response that is just the initial Facebook feed at the time that you requested it. Then you’re going to see other messages pop up on your feed interactively, whether this is Facebook or Twitter. Or better yet, maybe you’ve asked for a map and maybe you’ve moved or scaled your map so that you are interacting with a page on a pretty fundamental level, and there are updates that are happening.
So at this point, a full stack developer is not making a request and responding to a request anymore. That’s not the full job description. Think about sending the request that’s baked into the browser, and think about presenting the request that’s baked into the browser. So all the rendering, all the sending and receiving. In a very real sense, old-school full-stack developers just needed to connect to that initial application. Really, it wasn’t a distributed application. They could do all the development and keep their head in one place.
Now, today’s full-stack developers work with truly interactive applications.Those interactive applications can easily become distributed applications. That part of the application has to be written on the client in JavaScript, and part of those applications has to be written on the server in some other language.
The thing that programming LiveView does is it kind of reins in this idea of what it means to be a full-stack developer in two different ways. The first one is that developers don’t have to build distributed applications anymore on the main, for the most part. The second thing that it does is it doesn’t force developers to write as much JavaScript. That means that they don’t have to be a multiple language developer anymore, and that’s a tremendous boon for productivity.
“The thing that programming LiveView does is it kind of reins in this idea of what it means to be a full-stack developer”
In a functional language, a full-stack developer is much like it is in other environments. When you think about it, request response programming or communication-style programming, those are effectively functional problems. You could think about a single web request as one giant function. You provide a URL and it provides a response. That will get you through request-response–style applications, but it doesn’t really get you through this idea where you have events flowing into the webpage, events flowing out of the webpage, events flowing into the server, and page fragments flowing out of the server.
What functional programming applies to that particular problem is that it allows us to break down problems and put them back together and compose them in a way that allows us to solve a small part of the problem at any given time.
For example, when I’m working with Phoenix LiveView, I don’t have to consider both the rendering problem and changing data based on inbound requests. So for example, when I’m working on a Phoenix LiveView application, I might hard code the state of my particular application and put a list of users.
Just hard code that, put that into my data bank. So I take my data bank and hard code a bunch of users in there. Maybe I have one that’s User A, maybe I have one that’s User B. And so, there’s this list of users. And I’m going to send this list of users through a function that says, “Hey, build me a web page.” That’s all I have to do when I have my mental state focused on building the web page.
Then when I’m done with that work, I can say, “Hey, what happens when the user clicks a link?” Add a user. And so, maybe the “Add a user” comes up with an additional user, User C. And so, all I need to do at that point is add the C user to my data bank. I don’t have to think about how that’s going to be rendered. When I’m dealing with events, all I have to do is deal with my data bank and the events. When I’m dealing with a web page, all I have to be dealing with is the web page and rendering the web page.
That’s all about building composable problems with different functions. And that’s really one of the things that functional programming is very good at.
“That’s all about building composable problems with different functions. And that’s really one of the things that functional programming is very good at.”
How has LiveView been influenced by and, pardon me, reacted to React-style development?
Is LiveView taking advantage of this basic idea? The answer is yes. This is effectively an event sourcing–based system. What that means is that when an event comes in, you have one dimension of the problem to think about. What does the event do to your data bank, to this kind of set of data? How does it impact the data?
This declarative set of rules, all that means is that when an event comes in, this is what you do. You build the rule, and whenever this data bank changes, this is what you do. And one of the things that you do is send down a list of changes to the user that is stitched into whatever you’re building, whether it’s a mobile application or something that’s on the internet of things. You just send down changes.
Then on that client system, it knows how to take those changes and weave it into whatever interface or whatever experience you’re building, whether it’s a thermostat or a car or a browser or a mobile desktop. This is called event sourcing, and this is basically what React is.
React is written in JavaScript, but there are a number of functional languages that work on the same idea. One of them I wrote about in Seven More Languages in Seven Weeks, in a programming language called Elm. That one was really talking about building a programming system in a strongly typed way that deals with this event sourcing model.
I think that this is one of those ideas that started in functional programming spaces and kind of flowed out to things like React where it gained popularity. Elixir has seen tremendous advances with this idea based on pretty much all of the major abstractions, that one that we talked about called OTP, the user interface programming one that we talked about in LiveView.
There will be another one that I think that Dockyard is working on based on building user interfaces for mobile devices. Then there’s also a Nerves-type system that you’ll see many of the same ideas follow, but the idea is that you separate policy and rules from individually laying them all out. And so you build rules for how to deal with data based on input and how to render that data based on input.
How expansive is the Elixir ecosystem? It seems to be moving far beyond just the web into hobbyist platforms like Raspberry Pi with Nerves and onto the desktop with a variety of solutions like LiveBooks and into the internet of things.
I think that there’s something important in curiosity that we shouldn’t move away from. It is important for us to be learning the right systems for solving individual problems. One of the things that I did for a while is learn languages that are in the areas of important problems that I see.
One of those languages is the Julia programming language because it’s so good at expressing machine learning concepts in an interesting and functional way. But one of the things that was interesting was as I was doing that research, and as I was applying myself to that problem, I started seeing more and more machine learning things show up in the Elixir language. And one of the things that you asked was, how flexible was this system?
I think that there’s a couple of answers to that. One, is it’s not a perfect system for everything, but it’s a very good system for building systems. What that means is that I can have something like an Elixir, and I can apply that to an ever-growing set of systems.
“[Elixir is] not a perfect system for everything, but it’s a very good system for building systems.”
And as I hit the wall in those external systems, I can pull in just enough of the outside world to adapt what I’m building. And there’s already a couple of places that we’ve seen some of that happen.
One of those is… You mentioned the internet of things. Well, the Elixir Nerves package isn’t exactly like writing native Elixir, but it’s getting closer and closer to that. The other thing that we’ve seen is this machine learning framework, which is called Axon, where everyone always said that Elixir wasn’t very good at number crunching.
Well, what the Elixir machine learning team did was take this configurable array that was good at number crunching, and integrate that with the Elixir language and Elixir tools, and supplement that with tools and customizations, until it was a great way to build distributed systems or calculate them.
One of the things that came out of that effort is that once that team started to solve those kinds of problems, they started to see the benefits of this number crunching in Elixir. More importantly, we saw better ways to think about and express things that were traditionally done in object-oriented languages so that we could take models for image recognition and text prediction and integrate them—not just in a way that we can think about and reason about the overall machine learning models but also integrate them in ways that we can easily consume them in our existing Elixir projects. That’s what this project called Bumblebee is about.
So your question about are we building something that is flexible and adaptable and why is that? I think the answer is yes. And why is that? Well, we’ve built a system that is good for building systems, and that’s not an accident, right? Because José is a fan and a follower of programming languages, and he leaned on languages, like Closure and Haskell, that do these types of things really well.
In a jobs environment right now, which is becoming more and more unstable for people who are developers, what is the outlook for employment and job hunting using the Elixir community set of tools?
I would say the very first thing that should be important to people is make sure that they have marketable skills and make sure they know how to learn. More than anything else, programmers have to know how to learn. And so I would say if you are not happy where you are, and if you’re looking at building marketable skills, well, Elixir is definitely one of the most highly paid languages.
Also, it might be worth first looking within your own communities and finding places that are underserved and stepping into those first, especially in this struggling economy. The second thing is that if you are looking for something that is going to be a little bit more marketable for the long term, I would definitely start with functional languages.
The reason for that is that a lot of the problems that we are trying to solve, like concurrency, and working with increasingly powerful abstractions, will help us solve the problems of increasingly distributed web systems that we have to build. Problems like building applications within the internet of things must be reliable and must be self-healing, and building systems that are machine learning–type systems.
Elixir is a great place to get involved in those areas, and it’s one of the most highly paid programming languages. I think that there was a study that said it was the sixth-highest paid language, and there have been other listings in the past that have said it’s been as high as the third–highest paid language. So I do think that Elixir has a pretty good outlook, but I think ultimately, the thing that you have to do as a programmer is to be able to learn and expose yourself to programming languages and keep up your learning journey. That’s something that you have to do for yourself. You can’t rely on your company to do it for you.
How can Elixir help people advance from being a developer who works on production maintenance to something more?
I think the most important thing is, if you’re going to be a value chain developer, someone who is higher up on the value chain, you have to be willing to use the tools that are the most productive.
“I think the most important thing is, if you’re going to be a value chain developer, someone who is higher up on the value chain, you have to be willing to use the tools that are the most productive.”
One of the things that I like about Elixir and other functional languages is that these languages are at a higher level of abstraction, and that means that each line of code that you write does more work. The other thing is that those high-abstraction languages are easier to read about, so that code that comes out of those systems, if it’s well designed and well reasoned, that code is going to be easier to maintain, but you need to be able to learn to do that work yourself.
So I think the question is, what is an iterative step that you could take toward being an Elixir developer, or toward being a functional developer, let’s say? The first part of that journey is to actually value your career development.
That’s going to be really, really important. And some of that is actually getting your company on board with making changes to your career that are going to serve you well over time. Getting that buy-in is incredibly significant, because without that then you’re really limited to time that you have on your own. And so the questions about tangible steps that you can take are first, you can look for a project that you really support and that needs help and start working some tickets on that platform, and start asking questions.
Very often, some of the highest-value learning tickets that you can write are around the documentation, because those explain the abstractions and the way things work. And those of us who are closer to the bottom of that food chain, that don’t fully understand how these abstractions work, can really make tremendous improvements in the framework that they’re working on by improving that documentation and helping other people come along in the same journey.
And that’s effectively what I did with the Seven Languages and Seven Weeks book. I was really afraid, and I was trying to move my career by researching what to do when the programming languages that I was invested in changed. I basically said I’m going to start doing some research on functional languages, finding why they’re different from languages like Ruby and IO or JavaScript, which was the closest analog.
Then somebody told me that this was a valuable journey and I should bring people along with me as I wrote the story. Well, you can do the same thing by taking the knowledge, by your exploration into the frameworks that you encounter, into Stack Overflow, or any other place that you want to document your experience so that other people can take the same journey that you did.
If people want to learn, start learning Elixir, what are some great resources for connecting into the community?
There are a number of great ways to get into the Elixir community, and it really depends on what your interests are. If you’re a machine learning person, well, there’s a new machine learning community that’s getting off the ground, and that has the underlying framework called NX, that has the machine learning framework that sits on top of that, that’s called Axon, that has the kind of packaging, the concepts and models that are built in Axon, that’s called Bumblebee.
There’s also a number of other communities that you might find interesting. If you find that you like to tinker with things, the Elixir Nerves community has a number of places, a number of projects that you can try. And actually, there’s a couple of Prag books called Building a Binary Clock and Building Weather Station with Elixir Nerves that you can play with. And each one of those groups has a lot of work to do, and a lot of opportunities to interact with people.
There are meetups that are pretty prolific in the Elixir space. A lot of them are still meeting online since the pandemic. Not all of them. There are a good number of local meetups that you can check out.
The Elixir core documentation is excellent. And so whether you’re doing the elixirlang.org material, which I think is still in sort of a document or book format, but eventually we’ll be moving towards an interactive livebook, that stuff is great.
The Pragmatic Bookshelf has the best grouping of books in this area. And a couple of books that you might like are Programming Elixir by Dave Thomas. If you want to take the next step, you probably want something like Designing Elixir Systems with OTP.
If you want to learn to use Elixir on the web, you can pick up Programming Phoenix LiveView by myself and Sophie DeBenedetto. There are also a number of individual projects you can check out that will depend on your interest.
And the best way to learn Elixir is to ask people from those communities where they need help. And if you’re looking for where the Elixir people hang out, you can check out the Elixir forum, or there’s also an Elixir Slack group. And both of those are active, and they’ll tell you where to go for people that need help or for good places to learn.
How can people keep in touch with you, follow what you’re doing, and learn what’s coming up next for you?
I am active on Twitter, Mastodon and LinkedIn. I am @redrapids on Twitter. I am at redrapids@genserver.social. And that’s actually an Elixir implementation of Mastodon! Keep up with our Elixir training materials we are creating at Grox.io and our YouTube channel.
