Posted on

Unit Tests Equal Awesome

I’m working on a hobby project iOS app that lets me track my comic book collection. I’m interested in comic books because all these super heroes from my misspent youth rule the world of popular culture. While the cool kids were playing sports and going to parties I stayed at home reading comic books. In college I stopped and found other things to do (computer programming, talking to humans, MTV). But now in the September of my life comic books are back and grip our imaginations tightly with their mutant powers.

I wanted to get back to the source. Where did all this cultural power come from? As I started buying physical comics again I realized I needed to track these objects of my affection on my phone. And I bet there are already dozens of apps that do this but I like to create my own tools.

Book Binder is the app and you’l find the code on GitHub.

Book Binder is an iOS app with a web backend. It’s an enormously long way from finished. I have lots of parts of it to figure out. The two current big problems are that comic book publishers can’t count and the number of comic books published is huge.

Comic book publishers can’t count!

Let’s take the case of Daredevil. One of my favorites as a teen and now a big show on Netflix. For reasons that are beyond comprehension (probably marketing) Marvel has restarted the numbering of the “man without fear” 6 times! Daredevil #1 was published in 1964, 1998, 2011, 2014, 2015, and 2017–and I don’t mean republished (that happens too). Daredevil #1 from 1964 is a completely different comic book from all the other Daredevil #1s in the five succeeding years! At one point Marvel tried to fix the problem with “legacy numbering” and that’s why the current series of DD started with #598 in 2017 instead of #29. I have no doubt in my mind that Marvel will start over with Daredevil issue #1 soon.

The other counting problem created by comic book marketing is variant issues with different covers. The most recent issues of Doctor Strange may or may not be published with different covers for the same issue. Collectors apply letters to each variant but Marvel doesn’t seem to have official variant designations. I have Doctor Strange #2 variant edition, legacy #392, second printing. I’m not sure how many variant editions were published or what the variant letter for each edition should be.

This counting (really identifying) problem makes it hard to come up with a good data structure for storing a comic book collection. I’m using a combination of a URI (unique resource identifier) and JSON (JavaScript Object Notation. This way I can easily share data between the iOS app and web server and with other comic book collectors, sellers, and buyers.

The number of comic books published is huge!

How many issues of Daredevil or Doctor Strange have been published since the 1960s? It’s hard to say. I estimate between 400 and 500 for Doctor Strange but I’m probably not including annuals, specials, team ups, side series, and all the variants. So let’s double that to 800 to 1000. And that’s the “master of the mystical arts” alone. If Marvel has around 200 books and DC has the same then we’re looking at a lower bound of 320K and an upper bound of 400K just for the two majors. Some of DC and Marvels comic books started in the 1930s and 1940s. If we include those and all the indy publishers (like Dark Horse) and all the publishers who have disappeared (like EC) then I’m going to estimate 1.6 million to 2 million unique comic books published in the USA. It’s really hard to say because it’s hard to know where to draw the line with publishers and if certain reprints should be included.

In any case I’m not going to be able to store more than a fraction of the millions of published comic book metadata representation in a phone. At best I can store a slice of this data locally and using any one of the big clouds to keep a shared catalog. I just want all this info to be quick to access, cheap to store, and easy to reconcile.

Testing an app for that

Let me tell you, creating an app, on my own, as a hobby project, is fun but hard. Like climbing a rock wall (which I would never personally do) you make a lot of false starts and have to retrace your steps trying to find a path forward.

This is where my unit test have helped. No, not helped. Made everything possible!

I started with three or four data structures. I’m testing out ideas and changing my mind as the idea do or don’t pan out. I’m not afraid to make large scale changes to my code because every function of every class has unit tests to make sure that if I break anything I can fix it.

Today I realized I had to take a big step back. I could not instantiate a comic book collection from a list of comic book URIs. I also realized I was storing state info in the comic book URIs which would not scale with millions of books to track. I finally realized that I had to enforce consistency in the formation of my comic book URIs (they all have to have four slashes). This way I could tell if a URI was mangled or incomplete.

I had to touch every one of my six major object that support my app… And I did! With Confidence. Once I removed state from my URIs and got all by unit tests to pass I fired up the app–and it worked fined. I had not added any bugs or broke any functionality. Whew!

If I didn’t have unit tests I’d be afraid to touch the code. I would be much more respectful of the code and I once I got some part of it to work I’d leave that part alone. As this is a lonely hobby project, I’d get stuck, give up, and move on to something easier.

Even with commercial software, with large teams of expert programmers, lack of tests and fear of changing the code, results in most software projects falling behind, abandoned, or just buggy.

I was sold on unit tests and Test Driven Development before and I’m resold every day I write code. I don’t care if you write the tests before or after the code that makes them pass (I do a bit of both). Just write the tests–especially if you are writing code for self-driving cars or robot military machines.

Posted on

iPhone to the Max

I’m on that Apple program that where you pay for an iPhone over time and you get the opportunity to update immediately when a new model comes along, as it does every year.

While this is a very good deal for Apple, almost like a subscription service, it’s a good deal for me too. I hate the feeling of FOMO that comes along when a new computer, phone, or device is released. But with iPhones (and Android phones) it’s more than just a feeling. Missing out on the latest phone means missing out on important new features, security protections, and performance improvements.

FOMO used to be a big problem for personal computers as there were big performance jumps between PC models back when Moore’s Law was still in full effect. These days you can still get great results from a 5 year old PC or MacBook. Maybe you can’t play VR games but you email and browse the web like a champ.

Smart Phones are in a different place on the product evolution curve than PCs. They still have a long way to go before they settle down. Innovation in smart phones is driven by advances in displays, cameras, custom chips, and machine learning. Even incremental improvements in these technologies means far better user experiences, security, and even more epic cat photos.

I’m super happy with the jump between the iPhone 8+ and the XS Max. It responds faster, is easier on the eyes with its 6.5″ screen, and yet is basically the same size as the 8+. The name is kinda of silly. But I don’t care what Apple names their phones.

At some point phone hardware will cease to evolve and some new device will become our “primary interface” to the Internet. My guess is that it will be a watch of some sort with accessory glasses or screens. But these things are hard to predict.

I’m on the train as I type this post into my phone and one guy is still reading a paper news paper. I guess that iPhone XS Max just didn’t excite him.

Posted on

Is It 1998 Again?

Set the Dial to 1998

Let’s power up the time machine and take a quick trip back to the wide world of tech around 1998. Microsoft was the Khaleesi of software and controlled a vast empire through Windows, Office, and Internet Explorer. Microsoft marched its conquering army of apps over the desktop and through the Internet with innovations like XMLHttpRequest, USB peripherals, and intelligent assistants.

All three of these innovations would go on to fashion the world we live in today with websites that look and feel like apps, devices that plug and play with our computers and phones, and helpful voices that do our bidding.

But back in 1998 these groundbreaking technologies were siloed, incompatible, and unintuitive!

  • You’d find that fancy web apps were usually tied to a specific browser (Walled Garden).
  • You’d buy a USB mouse and often find that it couldn’t wiggle your pointer around the screen (Standard Conformance).
  • You’d grow frustrated with Clippy (aka Clippit the Office assistant) because the only job it could reliably do was “Don’t show me this tip again.” (Poor UX).

And this is exactly where we are in 2018! Still siloed, incompatible, and unintuitive!

  • Do you want to run that cool app? You have to make sure you subscribe to the wall garden where it lives!
  • Do you want your toaster to talk to your doorbell? Hopefully they both conform to the same standard in the same way!
  • Do you want a super intelligent assistant who anticipates your every need, and understands the spirit, if not the meaning, of your commands? Well, you have to know exactly what to say and how to say it.

Digital Mass Extinction

The difference between 1998 and 2018 is that the stakes are higher and the world is more deeply connected. Products and platforms like Apple’s iOS, Google’s Cloud IoT Core, and Amazon’s Alexa existed in 1998–they just couldn’t do as much and they cost a lot more to build and operate.

In between 1998 and 2018 we had a digital mass extinction event—The dot com bubble burst. I was personally involved with two companies that didn’t survive the bubble, FlashPoint (digital camera operating system) and BitLocker (online database application kit). Don’t even try to find these startups in Wikipedia. But there are a few remains of each on the Internet: here and here.

Today, FlashPoint would be reincarnated as a camera-based IoT platform and BitLocker would sit somewhere between iCloud and MongoDB. Yet the core problems of silos, incompatibility, and lack of intuitive control remain. If our modern day apps, IoT, and assistants don’t tackle these problems head-on there will be another mass extinction event. This time in the cloud.

How To Avoid Busting Bubbles

Let’s take a look at the post-dot com bubble burst world for some clues on how to prevent the next extinction. After the startups of the late 1990s died-off in the catastrophe of the early 2000s the designers, developers, and entrepreneurs moved away from silos, proprietary standards, and complicated user experiences. The modern open standards, open source, and simplicity movements picked up steam. It became mission critical that your cool app could run inside any web browser, that it was built on battle tested open source, and that no user manuals were required.

Users found they could buy any computer, use any web browser, and transfer skills between hardware, software, and services. This dedication to openness and interoperability gave great results for the top and bottom lines. Tech companies competed on a level playing field and focused on who could be the most reliable and provide the highest performance and quality. Google and Netflix were born. Apple and Amazon blossomed.

Contrast that with the pre-bubble burst world of 1998 (and 2018) where tech companies competed on being first to market and building high walls around their proprietary gardens.

If we want to avoid the next tech bubble burst (around 2020?) we need Apple, Google, Amazon, and even Netflix to embrace openness and compatibility.

  • Users should be able to talk to Google, Siri, and Alexa in exactly the same way and get similar results (UX transferability).
  • Users should be able to use iOS apps on their Android phones (App compatibility).
  • Users should be able to share connected and virtual worlds such that smart speakers, smart thermostats, and augmented reality work together without tears (Universal IoT bus).

Google and Apple and Standards

At Google I/O last week the Alphabet subsidiary announced a few of examples of bubble avoidance features…

  • Flutter and Material Design improvements that that work as well on Android as they do on iOS.
  • AR “cloud anchors” that create shared virtual spaces between Android and iOS devices.

But sadly Google mostly announced improvements to its silos and proprietary IP.  I’m sure at the WWDC next month Apple announce the same sorts of incremental upgrade that only iPhone and Mac users will benefit from.

Common wisdom is that Apple’s success is build on its proprietary technology from custom chips to custom software. This is simply not true. When I was at Apple in the 1990s success (and failure) built on a foundation of standards, like CD-ROM, USB, and Unicode. Where Apple failed, in the long run, was where it went its own incompatible, inoperable, way.

In the 1998 the macOS was a walled garden failure. In 2018 macOS is a open source BSD Unix-based success. More than Windows, more than ChromeOS, and almost as much as Linux, macOS is an open, extensible, plug and play operating system compatible with most software.

The Ferris Wheel of Replatforming

Ask any tech pundit if the current tech bubble is going to burst and they will reply in all caps: “YES! ANY MOMENT NOW!!! IT’S GONNA BLOW!!!”

Maybe… or rather eventually. Every up has its down. It’s one of the laws of thermodynamics. I remember reading an magazine article in 2000 which argued that the dot com boom would never bust, that we had, through web technology, reached escape velocity. By mid-2000 we were wondering if the tech good times would ever return.

Of course the good times returned. I’m not worried about the FAANG companies surviving these bubbles. Boom and bust is how capitalism works. Creative destruction as been around as long as Shiva, Noah, and Adam Smith. However, it can be tiresome.

I want us to get off the ferris wheel of tech bubbles inflating and deflating. I want, and need, progress. I want my apps to be written once and run everywhere. I want my smart speaker of choice, Siri, to be as smart as Google and have access to all the skills that Alexa enjoys. I want to move my algorithms and data from cloud to cloud the same way I can rent any car and drive it across any road. Mostly, I don’t want to have to go back and “replatform.”

When you take an app, say a banking app or a blog, and rewrite it to work on a new or different platform we call that replatforming. It can be fun if the new platform is modern with cool bells and whistles. But we’ve been replaforming for decades now. I bet Microsoft Word has been replatformed a dozen times now. Or maybe not. Maybe Microsoft is smart, or experienced, enough to realize that just beyond the next bubble is Google’s new mobile operating system Fuchsia and Apple’s iOS 12, 13, and 14 ad infinitum…

The secret to avoid replatforming is to build on top of open standards and open source. To use adaptors and interpreters to integrate into the next Big Future Gamble (BFG). macOS is built this way. It can run on RISC or CISC processors and store data on spinning disk or solid state drives. It doesn’t care and it doesn’t know where the data is going or what type of processor is doing the processing. macOS is continuously adapted but is seldom replatformed.

To make progress, to truly move from stone, to iron, to whatever comes after silicon, we need to stop reinventing the same wheels and instead, use what we have built as building blocks upon which to solve new, richer problems, to progress.

 

Posted on

Future Shock Reloaded

When I was a lad I would sneak into my father’s den and ransack his library. This was the early 1970s and I discovered all sorts of treasures, many of which I could barely comprehend, including The Godfather and even The French Lieutenant’s Woman. But the one book my pre-teen brain could parse was Future Shock by Alvin Toffler. I didn’t read the whole thing, too many pages! But I got the gist: “too much change in too short a period of time” is bad for us.

Toffler argued that human beings and culture just can’t keep up with the accelerating rate of modern technological change. For thousands of years we were farmers; for hundreds of years we were crafters; for decades we were factory workers, and now, what are we?

  • Last week we were web developers worried about search engine results.
  • This week we are gig-economy drivers living on apps for pickups.
  • Next week we’ll be data analysts mining Bitcoins on behalf of AIs.

Indeed, we’re getting those accelerating returns that Ray Kurzweil promised. Every week there is new technological anxiety adding to our already overwrought imaginations. Future Shock is back!

At home, at work, and on the road, I run into people worried about the impact of technology disruption on their careers, their health, and their spirits. And people have questions. Urgent questions that are difficult to answer but critical to understand in our Future Shocked world…

  • “Should I buy Bitcoin?”
  • “Will Robots steal my job?”
  • “Should I plug in an Alexa and allow it to spy on me?”

These are all great questions. And I don’t really have the definitive answers. But I have some ideas. The future is hard to predict but there are trends that we can look for and some basic laws of human behavior that we can count on. I’m going to share my answers below. I’m probably wrong but these answers are the best advice I can give.

Should I buy Bitcoin?

If you have to ask the question then the answer is probably “No.”

Buying or mining any cryptocurrency was a fun hobby in 2009. Today, Bitcoin and its friends Litecoin, Ripple, Dogecoin, Ethereum, et al. are basically a modern form of gambling and money laundering. If you have money you can afford to lose, and you’re not already in Las Vegas, go ahead, gamble on Bitcoin.

What we should buy as in buy into is Blockchain. It’s the future of how we’re going to contract with each other. There is a great book entitled Debt by David Graeber which explains that money is a social debt or obligation contracted between two parties. And these social contracts existed long before dollar bills, bills of sale, and other formal written contracts. Blockchain is the modern incarnation of social contracts as software. Blockchain is decentralized and open and part of a larger trend of “software eating the world.”

I bet in a decade or two almost every thing of value, including the labor of people like you and me, will be tracked and traded via blockchain-enabled smart contracts. Partnerships, concert tours, college educations, and even cute little human babies will hold ICOs (Initial Coin Offerings) and we’ll all invest in each other such that the rising or lowering tide of value we collectively earn (or mine) will be used to pay our debts.

Blockchain will make the value we create for society as individuals *fungible*. I’ll be able to buy a block of your creativity and you’ll be able to buy a block of my coding skill.

Will robots steal my job?

Yes… but it’s ok, you’ll get another job (if you become T-Shaped).

There is no industry, vocation, or advocation that will not be impacted by automation and machine learning. In only a few more years RPA, Robot Process Automation, will automate entry level and managerial tasks. Everything from buying toilet paper to approving expense reports will be automated by a narrowly smart bot—not a physical robot like R2D2. Telemarketing, accounting, retail, sales, driving, and even acting and singing will be automated so that the labor is cheaper, cleaner, quicker, and more reliable.

And we want this world of smart, unbiased robots, lowering costs and improving service.

My favorite example is the Amazon Go Store. No cashiers, no money, no lines, no rudeness, and no shoplifting. Sooner than we think the major retail stores will be crewed only by a store manager or two while cameras and sensors do all the other jobs. Not only cashiers but security guards, cleaners, stockers, and customer service jobs will be automated away or performed remotely.

Luckily, in the world of software engineering, we have a great deal of experience with automation-driven employment disruption. When I started coding in the 1980s I used low-level programming languages like Assembly and C, but most of the work was the *housekeeping* required to build, test, and deploy software. Today, while I’m still programming, the real innovation is in DevOps and Cloud Computing and all the ways they automate the software development process.

Software engineers like me have learned to continuously learn! We’ve also learned to stretch and try new types of jobs every few years so we remain relevant and employable. In the software world we call this being T-shaped. This means we know a lot about a lot of things in our general domain but we also have a specialization or two where we have expertise and experience.

No matter what kind of expertise or experience that you have now, that enables your employment, it will probably be automated away in a decade or two. However, your domain will remain and new areas of specialization requiring new expertise and experience will open up, creating new jobs and new opportunities.

Should I plug in an Alexa and allow it to spy on me?

Absolutely.

Choose a voice assistant, Alexa, Google, Siri, and learn how to take advantage of it as it learns how to take advantage of you.

You’re worried about your privacy and security. Me too! But ignoring or unplugging technology won’t protect us from corporate snooping or hacking. The best defense of our privacy and data is strong engagement with and deep understanding of the technology that surrounds us.

Decades ago, I lived in a world where we had to type at a computer to get it to do anything. Today, I mostly touch screens to get stuff done. But typing and touching isn’t any more secure than a Home Pod that listens for and responds to my voice. And these smart speakers aren’t mind readers. We have to learn how to talk to a computer as it has to learn how to understand us. Both get better with practice!

What about privacy? Personally, I don’t think we have privacy anymore. That ship has sailed away in a sea of cameras and microphones built into everything. But we do have anonymity. For the most part, unless a shadowy government agency is out to get you, your personal data is lost in the noise of trillions of packets of data bouncing around the Internet.

Can we trust big capitalist enterprises like Amazon, Google, and Apple? Well, they are big enough to be held legally accountable and regulated. They are interested in our money, not our souls. So trust.. but verify!

How I learned to stop worrying and love Future Shock

The world works by each of us influencing each other. The whole history of computer-enabled communication is just the most recent installment of the ancient arts of oratory and rhetoric. “Friends, Romans, and countrymen, lend me your ears” is the invocation of a conversation. Just like Amazon, Google, and Apple, Shakespeare is trying to influence you. Shakespeare had an agenda. Shakespeare was selling something but he was also listening for what you had to say.

I know that this is sometimes hard to believe, but the overall trend in the USA is that people are healthier, living longer, and generally wealthier, even when we consider all the conflicts, violence, and greed in the world. In the Neolithic Era we lived to 25 and died in the jaws of a tiger. In the Middle Ages we lived to 35 and died at the end of a lance. In the 21st century we live well into our 80s and have the Netflix Skip Intro Button. That button alone gives me hope!

It is technology that is making the difference: all that technology that we are anxious about; all that technology creating Future Shock.

An electrical shock only happens when you make initial contact with a charged wire. Don’t wait for the charge to build up. And once you make contact, one you embrace the future, stay connected so you can influence and own the future.

Posted on

Meeting Madness

It’s only getting worse. And there is no cure. I’m talking about meetings. Most sapiens are natural social networkers. And that means we need to know what is going on, who is doing what, how it’s going to be done, where we’re doing it, and why we’re doing it this way and not that way. Often the meetings take up more time than the task.

I have over 30 years of meeting experience and I feel confident that we can do better. Here’s my advice for optimizing meetings, both from the point of view of an attendee and as an organizer.

Attendee

Look at your calendar! Back to back meetings. Double and Triple booked. No time for lunch or travel between meetings. No time for work or thinking. So you bring your laptop and work during the meeting, not paying attention until you are called upon. You bring your lunch and eat during the meeting. You find yourself day dreaming as you wait your turn to give an update for 5 minutes during a 60 minute meeting. You try to ask questions and argue points only to get dragged off topic as the meeting spirals out of control. I’ve experienced all this and I’ve done all this.

First, don’t skip lunch. Take 30 mins in the middle of every day to eat, meditate, or watch a video on clean coding. You need your energy to focus and participate.

Second, tell everyone who invites you to a meeting (even me) that you are short handed and likely to remain that way so they have to accommodate you (not the other way around). You need the option to leave early after your part is over. Sit by the door. Arrive on time and leave as soon as you can.

Third, request an agenda or list of topics that require your attention and attendance. Try to do the work and provide the info before the meeting and politely ask if you can skip it. There are only 131,400 working hours in an average lifetime. Make sure you’re not spending all those hours in meetings.

Organizer

Every once in a while I bet a co-worker has grabbed you at the end of a meeting and complimented you on what a productive and inspirational discussion you just led. What a great feeling! Every meeting you hold should meet that bar!

First, ask yourself if you really need this meeting. If you don’t cancel it. Even if it’s a regular status meeting. If everything is on track and you don’t need anything from anyone, kill the meeting. The best meeting is the meeting that never happened.

Second, plan your meeting. Supposedly there are six types of meetings: status update, info sharing, decision making, problem solving, brainstorming, and team building. Use this list to make sure you and your attendees are ready to have the meeting.

  • Status Update and Info Sharing: Tell folks they can skip the meeting if they get you the info you need before hand. Make sure you send out a report of all the collected status after the meeting and only those with something to contribute need to attend.
  • Decision Making: Provide the decision points and analysis before the meeting. Make sure you provide a report of action items and decisions made after the meeting so that everyone is on the same page afterwords–that way you don’t have to repeat the meeting!
  • Problem Solving and Brainstorming: Provide background beforehand and outline the process you’ll use to run the meeting. After the meeting write up the results and explain next steps.
  • Team building: Have fun and keep it short. Frequent, brief, fun activities build better teams than infrequent, all-day, somber off-sites.

Most meetings are combo meetings as in let’s share some status and then make a decision or let’s doing brainstorming and then some team building. I’m pretty sure combo meetings are a bad idea. Try to turn combos into separate meetings. If you can’t break them apart, provide a break between segments and a formal agenda so that attendees know which meeting-within-a-meeting they are in.

Third, direct your meetings. Every meeting should be structured like a great movie with a beginning, middle, and end. Every meeting should have a clear protagonist and antagonist. Every meeting should have a quest, that once achieved, signals that the meeting is over and everyone can go home (hopefully early). Keep your attendees alert and awake. Give them permission to leave after their part in the movie is over. Edit digressions and dissembling out of the movie. Ensure the meeting is meaningful to your attendees. That means stopping some people from chewing the scenery and calling upon other people to play their parts.

My Favorite Meetings

Every week I hold a staff meeting with my direct and some in-direct reports. I don’t care much for the org chart so the invitation list grows and shrinks as needed. Sometimes I do a great job and the staff meeting is truly like a movie. But much of the time I create a flop of a meeting. I’m trying to do better.

I’m inspired by the brevity and tightness of the meetings recommended by the Agile development process. I’ve found that, even if you are not writing code, a daily standup that lasts for 15 minutes and asks three questions (What have you done? What are you planning to do? Are there any impediments in your way?) works wonders for managing any sort of on-going issue.

The retrospective is another great idea to steal from Agile, especially for performance reviews: What is working? What isn’t working? What needs to change?

We spend so much time in meetings that I fear we’ve become complacent. If you’re anything like me, you mindlessly accept any meeting that comes your way and then wonder how you’ve completely run out of time.

Well, let’s stop the meeting madness. it begins with you and me!

Posted on

The Voices of Software Development

The other day I was writing up a job description, which I take rather seriously, and I remembered that I needed to include the statement that an engineering manager has the responsibility to function as the “voice of the engineer”.

Then I sat back for a moment and thought,

  • Is this true?
  • Why can’t engineers speak for themselves?
  • And what about other constituents in the software development process?

Let’s look at each of these questions in turn starting with the last and working our way up.

The constituents of the software development process include the engineers working on the problem, the code being written, the system that the code implements, the users who happily (or unhappily) use these systems, and the business that one way or another pays for all of this activity.

The business tends to be the most powerful entity in software development. If you don’t have a business, even if that business is free software, then you don’t have any of the other constituents. The business is the organizing principle around which everyone else is invited to the party. Without the business nobody gets paid, whether that payment is in dollars, bitcoin, downloads, or GitHub stars.

Thus the business has evolved elaborate techniques to insure its voice is heard by all team members loud and clear. We call these techniques Waterfall, Agile, Scrum, Kanban, etc. And yet, many times the voice of business gets lost in the drive to complete the project, hit the deadline, keep costs down, or please the boss. Many times, being the loudest voice in the room means you get heard but not necessarily understood. From my perspective, it’s the job of the project manager, product owner, and engineering manager to make sure the business’s voice is both heard and understood. That means not just driving the team to deliver but to ensure the right sorts of tradeoffs are made between time, cost, quality, and scope. Even if it drives business stakeholders to complain in angry voices!

The user’s voice is usually the hardest to hear—unless you have actual users on the team. Even then, those users might not be representative of all users, or even the average user. Before the web, we didn’t have a reliable way to talk to thousands or millions of users directly. We had to hear their voices though focus groups, surveys, and experts. The results were often products likes the Ford Edsel or Windows Vista. Whisper Down the Lane is a dangerous game when you’re developing software. Luckily, we have this World Wide Web and the ability to test theories about the user’s voice directly. Generally, product owners and engineering managers take ownership of listening carefully for the user’s voice in the din of conflicting use cases and market requirements. But still, we’re completely surprised by what the user says.

I want to talk about the system’s voice and code’s voice together because they are often confused. In software development, the code is not the end goal; the code is a side effect of a process to get to the goal; that goal is a well-functioning system. You have to listen very carefully to hear the voice of the system because it’s not a person (not yet). Product owners, engineering managers, engineers, and architects all have a big responsibility to quiet their minds and meditate on the telltale whispers of a happy or sad system. A happy system scales easily up and down, is readily extended, responds quickly, and doesn’t consume more resources than it needs too. A sad system is just the opposite. Unfortunately, most systems are sad because hardly anyone is listening to them. Instead most of engineers look myopically at the code. You have can great code and still have a lousy system. You have terrible code and have a great system. Look at any well-functioning software product and you’ll find many areas of poor code and hasty hacks.

To hear the voice of the system you have to write tests, compare log files, profile disk, memory, and CPU usage, and monitor everything. It’s hard, time consuming, and your stakeholders will get cranky because the they and user didn’t explicitly ask for all this system work. Just explain to them that the bestest, fastest, prettiest car in the world isn’t going anywhere without roads and fuel.

What about the voice of the code? I love code. I listen to it very closely. Even though code is a byproduct of a process and not what the philosopher’s call teleological, it’s one of my favorite parts of software development. Code and the ability to hear what it has to say is what separates the engineer and engineering manager from the rest of the development team. I like to say, I don’t care what the code does, as long as it does it well! But that’s not quite true. Code for code’s sake is like an award-winning marketing campaign for vaporware.

Between the engineers, their managers, and the architects, there are plenty of people worried about the code, its styles, its elegance, and its fitness. Still, like the voice of the user, the voice of the code can get lost in the mad rush to fix P0 bugs and hit the end of the sprint—usually with deleterious downstream effects on the system and those users that everyone is supposed to be centered around.

At long last, we come to the voice of the engineers, those people that the engineering manager is supposed to speak for and represent. Why can’t they just speak for themselves? After all, nothing gets done if the engineers don’t code it! (We can’t yet compile and execute Photoshop UX designs.)

In small organizations, the voices of the engineers are very audible.

In large organizations, the engineers need an authority figure, someone with gravitas and political power, to make sure their voices are heard and understood by everyone demanding their time and attention.

Even though engineers have a lot of options these days, we’re living in Edsger Dijkstra’s software crisis:

“The major cause of the software crisis is that the machines have become several orders of magnitude more powerful! To put it quite bluntly: as long as there were no machines, programming was no problem at all; when we had a few weak computers, programming became a mild problem, and now we have gigantic computers, programming has become an equally gigantic problem.”

With the cloud, we have infinitely gigantic computers and programming them has become an infinitely difficult problem!

To solve for that problem every business is hiring more and more engineers. But we know from Frederik Brooks that adding more people to a software project simply makes it more complex and harder to deliver.

Why is that? The more engineers, the more voices, the more discussions, the more POVs, the more time spent on negotiation, and the less time spent on listening to the code, the system, the user, and the business.

This is why I like Jeff Bezos’s formula for team size: the two-pizza team. In large groups the quiet people stay quiet and the outspoken people spend all the time talking instead of listening. If you can break up your software development work such a team of six people can easily handle it over a serious of iterations, then there is a good chance every voice on the team will be heard.

This is why the engineering manager, the architect, and the agile coach have to be the “voice of the engineer”—Not because engineer voices are ignored but because there can be so many engineers on a team that their individual voices get lost in the roar of the crowd.

A sharp-eyed reader will notice that I’ve linked the engineering manager to every node in this graph of voices. A good engineering manager is usually the only team member technical and experienced enough to hear all the voices and try to balance them out. Of course, not every engineering manager remembers to do this. Often, we get lost listening to the loudest voice or the voice we like the most (our own). The trick is to stop talking long enough such that you have the time to invite other constituents to speak.

Posted on

Emoji Tac Toe Opened Sourced

Happy Father’s Day!

 

To celebrate my 28th Father’s Day I’ve opened sourced Emoji Tac Toe. It’s actually not a big deal to anyone but me. It’s kinda of scary open sourcing code that you wrote alone and without first cleaning it up. But what the heck. If someone can learn something from this code, why keep it locked away. It’s already been on GitHub for a year. It’s not getting any prettier under lock and key.

You can find the source code at github.com/jpavley/Emoji-Tac-Toe2. And you can download the iOS app on the App Store at John Pavley > Emoji Tac Toe.

You can play Emoji Tac Toe on your iPhone, your iPad, and your Apple Watch. (As long as you are running iOS 9.3 or later.)

I guess I should chat a little bit about the code just in case you want to take a peek.

First

I plan on refactoring the code quite a bit. I want to basically refactor it so that the core is separate from the iOS implementation and I can port it easily to the web and to Android. Maybe Windows too. Who knowns! I’m going to start this process by adding unit tests and then by tearing it apart.

Second

I plan on updating the code for iOS 11, including Swift 4 and ARKit. I’ve been meaning to add multiplayer over BlueTooth and MessageKit capabilities. I also want to complete the tvOS and macOS implementations.

Third

The core code lives in the EmojiTicTacToe.swift file. Since there are more emoji than I can count I have cherry picked the 1100 that I wanted to include. This is still too many and I should cut it down further. It’s too many emoji because choosing which emoji to play with is difficult. I can’t use Apple’s keyboard user interface because I can’t restrict it to just showing emoji. And I don’t want to waste my time recreating Apple’s design. Also, this game is not about typing anything so a keyboard doesn’t make sense.

Instead I create an array of emoji and it works very well. iOS is great at dealing with Unicode.

Tic Tac Toe is an ancient game and simple. There only eight winning vectors. So, it’s easy to brute force and just check any board for the eight vectors.

As emoji are text it’s simple to translate a game board into a string and back. Interoperability with messaging and tweeting is free. This is why I love emoji! Rich graphics without the cost of image file management. Once day when operating systems allow custom emoji we’ll stop using PNGs and JPEGs altogether. On that day the web will be more fast and safe than ever!

Given the simplicity of the game, my AI is equally simple. When it’s the AI’s turn, I look for an open cell, look for a blocking move or look for a winning move using the eight winning vectors as my guide. Because tic tac toe is too easy to prevent absolute boredom I add a bit of random error into the AI’s thinking so that if the player is paying attention she can beat the machine.

Four

ViewController.swift contains iPhone/iPad specific code.

I found I needed some iPad specific code to avoid a crash when presenting Apple’s standard share UIActivityViewController. I did not open a radar.

I handle several gestures that I’m sure my players never discover but they are there none the less:

  • A long press on an emoji can trigger an attack if battle mode is enabled. A few emoji will do cool tricks in battle mode. There are several battle mode strategy functions that implement these tricks. My favorite is youWin which lets the other player win.
  • Panning up and down turns sounds on and off. That should be a standard gesture for all games!
  • A shake starts are new game with a random pair of emoji. This is the best way to start a new game as choosing particular emoji is a pain.

Five

NewGameViewController.swift contains the code for the game settings on the iPhone/iPad.

Originally, I had the iPhone and Watch Extension collaborate so that one could control the other. But the effort was not worth the reward. Now the two version are completely independent.

I use a UIPickerView with two components to enable the player to choose two emoji. It’s not bad at all if there were only 20 or 30 emoji. But it’s just too much spinning to find a particular emoji out of 1100!

If the user tries to choose the same emoji for player 1 and player 2 (or the AI) I detect that and have the UIPickerView jump to the next emoji. See ensureRowsAreUnique(component: row:).

To make finding a particular emoji a bit easier I allow the player to jump over groups of emoji in the UIPickerView by tapping on the labels for each player. I’m guessing nobody would ever find this feature but the labels are colored blue to indicate they buttons.

Six

InterfaceController.swift contains the code for a very simple version of Emoji Tac Toe that runs on watchOS. I actually like this version if the game best. No battle mode, no sound, no popovers, no choice of emoji. Just a single player game you tap out on your watch while waiting for the train.

Programming the UI for watchOS reminded me of my VisualBasic days! Each button view has it own handler function. No way to aggregate the touches and dispatch them with a switch statement!

Final Notes

All-in-all this code is pretty rough and need a lot of work. But it does work and hardly ever crashes. So that’s something. There is a half-finished tvOS implementation but I’m going to rethink it so don’t look at it!

I had to delete the sound effect that I didn’t create myself. Your build of Emoji Tac Toe will not sound like mine. But otherwise you are free, within the MIT License constraints, to do what you like with the code.

Posted on

A Short Note on Tolkien’s On Fairy Stories

I’ve been a fan of J.R.R. Tolkien’s Hobbits and Middle Earth since I first ran into Bilbo Baggins at a children’s library appropriately as a child. I remember falling in love with the tale and being somewhat disappointed that the other books of its type were not as deep or as rich as Professor Tolkien’s work. As I was a little kid, I could not articulate that disappointment. I only felt a general sense of something lost when the book was over, and no other book on the same bookshelf return it to me: not Wind In the Willows, not Charlie and The Chocolate Factory, and not even the Wizard of Oz. All great books but even a child I connected with some inner meaning inside the Hobbit that stood on its own and seems as real and primary as the mundane world in which I lived.

I’ve been on a quest, as it were, to find that something, which continues to this day. It’s present in all of Tolkien’s works, from the Lord of the Rings to the Silmarillion. I’ve found it in more recent works aimed at kids, including J.K. Rowling’s Harry Potter and Pullman’s His Dark Materials. But there is much more of that something in classic Science Fiction. Asimov’s Foundation has it. Gene Wolf’s The Book of the New Sun has it. Iain M. Banks’ Culture has an especially strong dose of it. The best works of Stephen King and Agatha Christie have it. So, this something is not constrained to Fantasy or Scifi. Even works of non-fiction, such as Yuval Noah Harari’s Sapiens, is strongly infused with whatever this something is. Heck, even philosophers like Plato, Wittgenstein, Heidegger, and Bernard Suits talk about it.

I’m really name checking here, but whatever this magic is it’s really quite rare, difficult to express, and almost impossible to capture. It’s a uniquely human product and shows ups unannounced and uncalled in works of art, musical compositions, theatrical productions, motion pictures, essays, and books of all flavors and genres.

This magic is a specific something, not a general feeling or vague concept. But you feel it in your bones while comprehending it with your head. It’s not the cheap tug of pity on your heart strings or the slight-of-hand of flashy conjuring trick. Artists, writers, doctors, lawyers, scientists, directors, actors, podcasters, and even computer programmers deliver it unreliability and it somehow speaks to us universally.

Personally, I believe the modern master of this magic something is Neil Gaiman and is present in everything he has ever written but no less or more so than his masterpiece American Gods. (I’m not sure yet if it’s presence in the TV series).

For me been impossible to define and yet instantly recognizable.

What is this mysterious, universal, sometime that I first glanced at as a child and I’ve been chasing ever since as a pastime. It’s mere escapism? It’s a misguided religious impulse? Is it a sense of meaning and inner coherence that the so-called real world of random punctuated equilibrium is sorely lacking?

Image how utterly stocked I was to find that Tolkien, the master himself, defined this magic something simply and plainly, in an essay entitled On Fairy-Stories written in 1939. An essay hadn’t read until this last week!

Before I go into a few key points that Tolkien makes in his obscure essay let me explain how I found it (or it found me).

I love a good podcast. And by good, I mean folksy, intimate, and long-winded. Since podcasting became a thing, thanks in large part to Sara Koenig’s Serial, many podcasts have become indistinguishable from mainstream radio productions. But these are not good podcasts. The quality of a podcast is inversely proportional to the its production values.

The great podcast are exemplified by Philosophize This!, The History of English Podcast, and The History of Rome.  Each of these is the efforts of lone amateur, a self-made wonk that a professional media outlet would never hire, using a home-made studio, for whom money is an urgent secondary concern, and for whom her primary concern can only be expressed by rambling on about her chosen topic out of passion and an imperative to share.

One such podcast is There And Back Again by Point North Media, which I’m pretty sure is the DBA of one guy: Alastair Stephens. There And Back Again is a deeply detailed look at Tolkien’s works, starting in chronological order, and planned to run for several years. Each podcast is a recording of a live online lecture where Stephens interacts with his fans on Twitter and YouTube while trying to get through his analysis of Tolkien’s works. Most of these podcasts are over an hour long and Alastair often spends 30 minuets discussing a single sentence! This untidiness is the hallmark of a great podcast: It’s here in the earnest solo confabulation of the caster that you get insights and gems that you would otherwise gloss over or never hear in more edited, polished production.

The brightest gem for me was Stephens exploration of Tolkien’s essay On Fairy-Stories. During four decades of loving Tolkien I didn’t know this essay existed. Remember, my search for the magical something that Tolkien brought to life inside me is a pastime not my mission. I’m not actively searching. Mostly, I’m filtering through the bits of information that pass under my nose in the normal course of living life and flagging anything that might smell of the something for later analysis.

You should stop now and read On Fairy-Stories. It won’t take long… (I linked it again so you don’t have to scroll up!)

So, what did Tolkien say about fairy-stories that helped me, and hopefully you, understand what this magic something is?

First, Tolkien reveals that the dictionary is no help in defining a fairy-story. This observation, I believe, is an admission that Tolkien is inventing the idea of the fairy-story, right here, in his essay. Before Tolkien, the fairy-story was a fairy tale and didn’t consistently contain the elements that Tolkien feels are essential. This is a pretty brilliant rhetorical technique of bending outer reality to inner concept by claiming the authorities and the collective culture of humanity didn’t get it right!

Second, Tolkien argues against the idea that fairy-stories are for children and that fairies in fairy-stories are little people. Tolkien assures us that children can gain something from fairy-stories in the same way adults can–by reading them and becoming enchanted. This idea of enchantment is different from the willing suspension of disbelief. Tolkien points out that you have to believe in the primary world for a secondary world to take hold. Note: there is a subtext in the essay that the primary world is the original, and that we, the thinking creatures of the primary world, are sub-creators, creating secondary worlds. And further, that this is humanity’s role in the cosmos because we have the power to create names. Tolkien was famously a Catholic, in a Anglican country, and it shows in his idea of primary and secondary worlds.

Tolkien wants his fairies, which he calls elves, to be as large of humans, because they are peers of humans. While he doesn’t point this out in the essay, humans, alone of all animals on planet earth, have no peers. We have only ourselves to talk to. Until the day we make contact with intelligent alien life, elves are as close as we can get to another species with a second opinion on the universe–even if we have to invent them in our imaginations!

Third, Tolkien goes on at some length to discuss the origin of fairy-stories and touches on the ideas of invention, inheritance, and diffusion. His main point is that a story always has a maker. He doesn’t mean owner, or even author. He means that at some point in time, some person first told that story which was then retold by other people, and changed in the telling, until we get to the fairy-stories we have today. The important point, for me, the recognition that fairy-stories are created, artificial objects. The critical something, that is at the heart of all true fairy-stories, was originally place there by a person. Fairy-stories are the products of people.

Fourth, Tolkien defends the fairy-story from the criticism that it is mere escapism. In a brilliant bit of reasoning Tolkien explains that those disapproving of fairy-stories are confusing them with entertainment that allow us to “desert reality”. Tolkien notes that it’s not fair to condemn the prisoner for “wanting to escape a prison”. Amen Brother! Plato, Wittgenstein, Heidegger, and Bernard Suits all agree with Tolkien on this point!

The human condition, which we all share, is the struggle with confinement of our culture, our class, our race, our gender, and our environment. The one problem poor and rich, liberal and conservative, all the races, and all the genders share is the fact that we are impression in temporary shells of flesh and the opinion we hold against each other (Sartre: “Hell is the other people”). The magic, mysterious, hidden, indefinable something, buried deep in the heart of a fairy-story, gives us a temporary escape into enchantment and the opportunity to bring a bit of that something back from the secondary world into the primary world. We get a chance to improve the human condition!

Fifth, Tolkien observes that the world of the fairy-story is just a reflection of our primary world. A reflection where mundane meaninglessness becomes strange and meaningful. Tolkien talks about the beauty of candlelight and ugliness of industrial age electric street lamps. I don’t quite agree with him as industrial age electric street lamps is now a whole esthetic (Steampunk) and not considered ugly. Tolkien and I generally part ways on this point, which is a present theme in all his books. that any technology more complex than 18th century farming equipment is unpleasant at best and an agent of evil at worst.

But I think there is a more profound insight here, that there is a platonic beauty that can only be discovered by looking at our primary world though the lens of a secondary world. Not a distortion but a focusing on the details that we’re taking for granted.

Tolkien expresses this idea of platonic beauty, which to mortal eyes is both lovely and dangerous, with the concept of Mooreeffoc. This term is new to me, but the concept is not. Moreeffoc is the word “coffeeshop” written on glass as seen from the wrong side.  It’s the idea that the fairy world is right in front of our noses, if only we could see it.

Sixth and finally, Tolkien invents a new term for the happy ending and explains that happy endings are crucial element of all fairy-stories. Tolkien’s word is Eucatastrophe, which means a joyous turn of events and is easily mistaken for Dues ex machine. In the Great Fairy-Stories, those written by Tolkien, Rowling, Gaiman, Freud, and Thomas Jefferson, the happy endings are not specifically happy, and cheap plot devices are not at work to bring about a sentimental conclusion.

When Frodo is about the toss The One Ring into the fires of Mount Doom, loses his battle with the Ring, and cannot bare to complete his mission, Gollum (spoiler alert), bites off Frodo’s finger and leaps, or falls, into the lava finally destroying the Ring.

The power of that eucatastrophe is that the War of the Ring has been lost at the final moment. All the struggle has been in vain. There is truly no turning back and no way for the cavalry to arrive. Evil has clearly and completely triumphed.

But then, after defeat has been realized, an unexpected and unasked for agent makes a supreme sacrifice and saves the day. Best of all, Gollum, is acting in character, he is not an obvious agent of good or the divine, and he has no intention of saving the day! He doesn’t have a change of heart of discover the good inside him. Gollum just wants his damn ring back!

In The Hobbit, Bilbo reframes from killing Gollum. Bilbo should kill Gollum. Gollum is scary, dangerous, sick, and twisted by evil. In a 21st century video game Gollum is exactly who it’s OK to kill. By all measures of human justice Gollum is the type of person you should kill–or at least lock up forever.

But Bilbo, this silly, almost powerless, hobbit doesn’t kill Gollum. Bilbo reframes, not because he isn’t in fear of Gollum, but because Bilbo is a hobbit, and hobbits don’t kill people. Hobbits are not heroes.

In the finale of the Lord of the Rings, this humble act of not killing, which seems foolish as Gollum a danger to Frodo, this is the act that saves the world.

And that is the something I first glimpsed as a child and now understand as an adult.

The ending of the Lord of the Rings is far from sweet. The hobbits and the world around them is saved but scarred and is eventually destined to fade. However, the core values of Middle Earth remain intact.

We’re all destined to fade. We can’t protect ourselves or our children completely from evil. We live a real, primary world, not a fairy-story or a utopia. But we can venture into secondary worlds, which in my book includes political theories and scientific models as well as works of fantasy, and bring something valuable back.

In this world, every day, a Bilbo is not killing a Gollum, and years later, at the last minute, a ring of power is destroyed and the world is saved. These moments are not usually televised and don’t make for viral new headlines:

“Father reframes from slapping son and later son remembers this and reframes from assassinating president in 30 years.”

“Memory of mother’s love prevents terrorist from blowing up building in city center 20 years laster.”

“Friend randomly calls friend on a Saturday night and prevents suicide planned for that evening.”

“Guy buys homeless man a cup of coffee at Penn Station and the probability of nuclear winter replacing global warming reduced by 1%,”

There are many names for these moments of empathy but I didn’t have a great name until eucatastrophe, a good disaster.

Posted on

Swift Programming: Filtering vs For Loops

The current version 3.1 has come a long way from the Yet-Another-C-Based-Syntax of the 1.0 version of Swift.

One of the best features of Swift is how functional programming idioms are integrated into the core of the language. Like JavaScript, you can code in Swift in several methodologies, including procedural, declarative, object-oriented, and functional. I find it’s best to use all of them all simultaneously! It’s easy to become a victim of the law of diminishing returns if you try to stick to one programming idiom. Swift is a very expressive coding language and it’s economical to use different styles for different tasks in your program.

This might be hard for non-coders to understand but coding style is critical for creating software that functions well because a good coding style makes the source easy to read and easy to work with. Sometimes you have to write obscure code for optimization purposes but most of the time you should err of the side of clarity.

Apple has made a few changes to Swift that help with readability in the long term but remove traditional C-based programming language syntax that old-time developers like me have become very attached to.

The most famous example was the increment operator:

In modern Swift you have to write:

As much as I loved to type ++ to increment the value of a variable there was a big problem with x++! Most coders, including me, were using it the wrong way! The correct way for most use cases is:

Most of the time the difference in side effects between ++x and x++ were immaterial, except when it wasn’t and it created hard to track down bugs in code that looked perfectly ok.

So now I’m used to typing += to increment values even in programming languages where ++ is legal. (Also, C++ should rebrand itself as C+=1.)

Another big change for me was giving up for-loops for functional expressions like map, reduce, and filter. As a young man when I wanted to find a particular object in an array of objects I would loop through the array and test for a key I was interested in:

Nothing is wrong with this code—it works. Well, actually there is a lot wrong with it:

  • It’s not very concise
  • I should probably have used a dictionary and not an array
  • What if I accidentally try to change o or objects inside this loop?
  • If objects is a lengthy array it might take some time to get to 12345
  • What if there is more than one o with the id of 12345?
  • This for-loop works but like x++: it can be the source of subtle, hard to kill bugs while looking so innocent.

But I’ve learned a new trick! In Swift I let the filter expression do all this work for me!

In that single line of code o will be the first object that satisfies the test id == 12345. Pretty short and sweet!

At first, I found the functional idiom of Swift to be a little weird looking. By weird I mean it looks a lot like the Perl programming language to me! But I learned to stop being too idiomatic and to allow myself to express functional syntax as needed.

For you JavaScript or C programmers out there here is a cheat sheet to understanding how this functional filtering works:

  • let means o is a constant, not a mutable variable. Functional programing prefers constants because you can’t change them accidentally!
  • The { } represents a closure that contains a function and Swift has special syntactic sugar that allows you to omit a whole bunch of typing if the function in the closure is the last or only parameter of the calling function. (Remember in functional programming functions are first class citizen and can be pass around like variables!)
  • $0 is a shortcut for the first parameter passed to your closure. So you don’t have to bother with throw away names like temp or i,j,k,x, or y.
  • .first! is a neat way to get [0], the first element of an array. The ! means you know it can’t fail to find at least one element. (Don’t use the ! after .first unless you are 100% sure your array contains what you are looking for!)

I’m working on a new project, a game that I hope to share with you soon. The game itself won’t be very interesting. I find that I enjoy creating games more than I enjoy playing them so I’m not going to put too much effort in creating the next Candy Crush or Minecraft. But I will blog about it as I work thought the problems I’ve set for my self.