Archive for the 'programming' Category

Software engineering vs. computer science vs. ?

It’s interesting that from time to time I have a discussion with a co-worker about the difference between a software engineer and a computer scientist. And I always get out of this conversation convinced that I’m actually neither. But before I talk about myself, let’s go for a short definition both (note: this is not a dictionary definition, it’s mine – so feel free to disagree):

Software engineers:

  • Have system in mind when building code
  • Tend to optimize for to reduce deployment risk
  • Decompose the problem into customer-requested features
  • Enjoy the challenge of design for integration

Computer scientists:

  • Have algorithms in mind when building code
  • Tend to optimize to reduce number of cycles to solve the algorithmic problem
  • Decompose the problem into algorithms that need combining and solving
  • Enjoy the challenge of complex problems that get them closer to proving that P != NP

Most software developers are know cannot be 100% characterized into any of those definitions. It’s dependent on the project and their background. For example, if you don’t know much of how to design a highly scalable multi-host system to generate unique numbers, you end up focusing your solution on making your internal code be clean, with the least amount of repetitions and separation of the algorithms (for example, you might think that i++ is too slow so you create a specialized code that does that much more efficiently).

So, when trying to categorize myself, I realize that, while I do display behavior that can be characterized as one of the two classes that I’ve described, I think there is probably a class missing:

Engineering scientists:

  • Have system vision/theory in mind when writing code
  • Tend to optimize to make the code look more like their system, even when it means using an algorithm that is much more complex than needed for the current problem
  • Decompose the problem into pieces that look more or less like the final theoretical system
  • Enjoy the challenge of, after all the added complications, still being able to finish a product

That’s about how I would characterize myself. I make no claims that I’m a good coder. I do care about the code I write, but I think I care much more about this vision of what should the world be if a system was built to solve all past, present and future needs for all my customers that sometimes being stuck to writing code is painful.

The Zune bug

I’m not sure if it’s really true, but it comes from a reputable source:

Cause for ZUNE leapyear problem

It’s kind of a funny bug, actually:

year = ORIGINYEAR; /* = 1980 */
while (days > 365) {
    if (IsLeapYear(year)) {
        if (days > 366) {
            days -= 366;
            year += 1;
    } else {
        days -= 365;
        year += 1;

I heard that it had something to do with leapseconds, but it actually was a simple problem with leap years. Very easy to fix too. And now I understand why Microsoft’s workaround was just to let it run out of batteries and wait until the day after the last day of the leap year and turn it on again (source). All you had to do is get out of this infinite loop for the last day of the leap year.

I’m so glad I’m not working on any software that gets deployed to devices in the wild that you can’t just press a button (or maybe a couple of buttons) and deploy a fix. If something like this happened to any of my software, somebody would be paged and we could find the bug, patch it and deploy it in less than an hour (depending on whether there are higher priority builds and deployments going on at the same time). Poor Zune-ers.

There is us in everything – can there be computers in us?

Today I spent some time watching a thriller, The Reaping and it made me think about why it’s sometimes so hard for some people to watch some of this movies. In my opinion, it all goes back to the way we can understand the world outside us: we put ourselves in the shoes of the people that we are observing. However, we don’t quite get that imagining us in a fantasy world where there are people out there planning on killing us might not be a good idea.

Minds never stop where they should, right? So after giving up in watching the movie (it’s not that it’s a terrible movie, but it was close to 1 AM and I probably should be going to bed to enjoy a long Sunday) I started thinking about intelligence and how one can simulate it. If most of our ability to understand people comes from this “self-projection” ability, how can we make an algorithm understand people? Are we algorithm too? Is this the best that it can ever hope to achieve?

Let’s consider that our computer is a dog. The dog looks at us and realizes that our face is pointing in the direction of a toy, but we don’t jump on it. So the dog “thinks”: well, if I was this person and was looking at the toy, the only reason I wouldn’t jump on it and bite it is if I’m sick. So maybe he is sick and I should be away from him. Quite restrictive conclusion.

That’s probably not what we are looking for when we think of computer understanding. So what can be done? Put all our learners in Second Life? 😉


So my current team is doing Scrum. We are on our 2nd iteration, so it’s something quite new, but I’m already seeing some signs that worry me. Before I get to them, though, I’ll document here some interesting reactions I’ve received from people when I mentioned “scrum” (in a completely different context – I had tendinitis not too long ago and I told my techie friends that it was the result of my fight with the scrum master):

“Oh, you do Scrum? I’m sorry… We do it too and the only interesting things we’ve delivered happened when somebody just got outside the iteration and worked on things in a scrum-less fashion”

“Are you doing Scrum? No? Great! Run away!”

“Scrum? Argh…”

Yes, these were real reactions of people that will be kept anonymous. So all these people had the experience and had their reservations. Why? Well, now that I have some experience with it I’m starting to get an idea why. But before I get to it, I’ll try to smooth it out by talking a little bit about what I think is great about it.

  1. Visibility: You know what is going on most of the time. If somebody is struggling to get something to work, you know at the time the struggling starts and not 2 weeks later when this person can’t deliver what they said they were going to.
  2. Nimbleness: Because things are sort of decided day-to-day, it’s easier to readjust plans midway. Things change all the time in most industries.
  3. Communication: Sometimes considered the most important aspect of Scrum (or any Agile methods). It forces you to discuss things with everybody all the time. It encourages people to argue and move as a group. This facilitates integration and sharing of information and experience throughout the team.
  4. Well-defined end: by forcing deliverables at the end of the iteration, it forces people to just get it done with. Considering a project finished, especially when it involves “fuzzy” things, like UI and operations support, is hard.

There is more I could add to this list, but I think these are the most important positive aspects. Now onto my concerns:

  1. Distancing of the specific person on the team to the project: because all projects are “shared”, each team member ends up losing that maternal connection to the code that is generated. It’s not something that is yours anymore. You lose the will to just stay late to get something extra done, as nobody else is doing it anyway. Your personal life might appreciate it, but I’m not sure this fulfills the coder inside you.
  2. Focus on deliverables generate internal operational mess: it’s hard to explain to a client why you chose to take 5 days to do something that you could have done in 2 just because you claim that this will make your life easier to deliver something that is only scheduled 2 months from now. Or because you think that if you choose the slower way you will get something that will be easier to deal with in cases of operational emergency, even though this might only happen 2 years from now. I think I’ve been through operational headaches enough (I was just on call) to know that the more you can build already thinking of operations on failure modes the better. You can never predict all failure modes, but covering as many as you can is the least you should do.
  3. Very long iteration planning meetings for something that is very likely to change in the future. This is aggravated if the team is quite efficient and has a lot of things that they can do in the iteration. You plan them all (for example, we have something like 9 projects planned this iteration) with 1-2-day long tasks, taking many hours of your month to do that and then, in about 2 weeks you find out that half of the things you’ve spent all this time to plan will be pushed out because something else showed up. Maybe this is more of a failure of my own team, but I just think sometimes there is too much of a granular task focus and building 50-100 granular tasks in a meeting is tiring and very error-prone. So if it’s going to be wrong, why do it this granular this early on?

Anyway, we’ll be moving on with it. We’ll learn with it, we’ll deliver a lot of things, and then we’ll adapt the process to whatever really makes sense to the problem space we are dealing with.

Playing with Processing

One of the things I’ve been trying to do is to get back to playing around with technology. Getting to know what is out there so that I don’t get stuck on always using the same solutions to different problems. Today’s technology was Processing. Processing is a programmable environment for interactive graphical interfaces. It’s used mostly for data visualization and a little bit of art.

Why did I end up looking at it? Well, I’m always interested in data visualization. I’m interested because it’s an incredibly hard problem to solve. I’ve worked in the past with some really good people in large dataset visualization and they made me observe how challenging it is. But, at the same time, how easy it is to see patterns when you just have a way to show the data to people. The smarter your visualization technique is, the easier it is to see the meaningful patterns. But even with a silly visualization, data has this natural ability of always exhibit some sort of organization.

So, what do I want to visualize? I’m not completely sure yet. I was thinking of doing something like Buzztracker or what OnlineJournalismBlog but probably focused on trying to see more connections of things and dynamics of the news. It’s going to be hard to get reliable timestamped information, but it’s something that I’ve always been interested.

Another possibility for visualization goes to applications at work. Currently we are doing multiple projects to cleanup the Amazon catalog, but once we deploy new cleanup rules, we have very poor visibility in what it’s doing and where. Having a good interactive visualization mechanism would potentially provide us with data that can be used to improve our algorithms and datasets.

What are my first impressions of the project? I actually liked it. I thought it was straight-forward to do some things and that many of their abstractions for user interaction (the event handling methods and the draw() method) are quite simple to use and can be quite powerful if used correctly. It’s fun how easy it is to write a program that allows you to just click on a screen and get it to draw different shapes depending on the button pressed.

I’m excited so far! More on this when I find more time to play around with it.

RSS My FriendFeed RSS

  • An error has occurred; the feed is probably down. Try again later.