Rough Book

random musings of just another computer nerd

Category: Perl

My data-recovery story

I was looking through wayback machine at snapshots of my website, when I came across one from 2005. It reminded me of something I had almost forgotten. At some point in 2005, the network card in my FreeBSD server started to die. I got myself a new card and set about replacing the dying one. I can’t recall why anymore, but I guess I had needed to disconnect the hard-drive at some point. I remember that after I plugged it back in and booted up, I was greeted by a screenful of terrifying error-messages. Something horrible had happened to the drive that held my home directory, my website source-code, and my database. I had lost about 6 years worth of posts and images on my website. My first instinct was to power down the machine to prevent anything more being written to the drive, which I immediately did. After that I think I tried a bunch of disk-recovery tools to try and recover my data. But this was difficult because the filesystem was UFS. I can’t remember if there were any UFS recovery tools at the time, or if I tried them, but I remember having tried almost everything I could think of.

Out of desperation, I think I finally decided to use dd. I started dumping the data from the drive using the lowest size-setting possible in dd (I want to say it is a byte, but I don’t really remember). I then piped this into a perl script that would examine each byte, looking for magic numbers. The drive had been corrupted so badly that there wasn’t even any trace of a coherent filesystem anymore. I knew that the data I was getting were most-probably fragmented, but I didn’t care at this point. I would guess the file-type by looking for magic numbers, and then I would start dumping that data into a file until I found an ending marker, or if the file-type didn’t have one, until the start of another magic number. I remember having various settings in the script so that I could tune its behavior, especially when dealing with false positives. My priority was to retrieve my pictures, website, programming projects, and database. For my source-code I only had to look for ASCII data. For pictures I looked for file markers for JPG, PNG, and GIFs. The database was difficult though, because I was using MySQL. By sheer chance, I had decided to take a SQL dump of my website’s database the day before for backup purposes (ironically, on the very drive that would die the next day). This was ASCII data, and so it was one of the first things my script found.

I ran this script over a couple of hours I think, and then for most of the next day for good measure. Then I began the tedious process of sifting through these files, weeding out false positives. All said and done, I retrieved a good chunk of my data. I think I got back around 80% of my pictures, and almost all of my code and website source. It was a scary few days, but I’m glad that my desperation drove me to try something like this!

How I got a medal from the Army for writing code

In 2005 my National Guard unit was deployed to Iraq as part of Operation Iraqi Freedom. My MOS (Military Occupational Specialty) in the Army was 92A, which is basically a logistics and supplies specialist. My job was to order parts for mechanics, pick them up, return old parts, manage HAZMAT, dispatch/return vehicles from missions, and handle licenses. I also did a few other things that I don’t remember right now. Anyway, at the time, the heart of this system was a tool called ULLS-G (Unit Level Logistics System – Ground). I say “at the time”, because shortly after we came back, ULLS-G was replaced by SAMS-E (Standard Army Maintenance System – Enhanced), which incidentally uses Oracle as a back-end database. Compared to SAMS-E, ULLS-G was a dinosaur. I had used it quite a bit, of course, having been in the Army for about 4 years by the time I was deployed. It was a complete pain to use it. ULLS-G was a DOS application (yes, MS-DOS) and most of the computers I used it on at the armory were only running DOS (this was circa early 2000’s so it wasn’t too uncommon to still see DOS systems around). By the time I was deployed most computers were running WinXP/2K or something like that, and so you could run ULLS-G in “MS-DOS compatibility mode”.

Read the rest of this entry »

CherryBlossom

I’ve created a project page for the CherryBlossom programming language. You can check it out here. The interpreter is written in perl.

Introducing CherryBlossom

Over the past month, I’ve been working on a new project. It’s called CherryBlossom, and it’s a way to write programs using haikus. Strictly speaking, CherryBlossom is a brainfuck analog. I actually spent more time writing the obligatory “Hello World” program in CherryBlossom than I did writing the interpreter for the language. The idea behind CherryBlossom is simple. Brainfuck instructions are mapped to words that convey the essence of the Brainfuck instruction. Of course, this is a little subjective and also a little abstract.

Ultimately, it serves as a way to make program code not just functional, but beautiful and artistic. Thus, we introduce a new criteria to programming. Your code must not only be elegant algorithmically, but must also be poetic and artistic (also, since program code consists of haikus, you need to represent your code in sets of 3 lines with the first and last lines having 5 syllables, and the second line 7. That is, conforming to haiku rules). CherryBlossom serves to blend the programmer and the poet into one entity (hopefully with amazing results).

Here is an example of “Hello World!” in CherryBlossom. I have opted to use a spruced up div tag instead of enclosing my beautiful poem in soulless sourcecode tags.
Read the rest of this entry »

bAdkOde: An Esoteric Language

I’ve added another project to the projects page. It’s called bAdkOde, an interpreter for an esoteric language that I designed. The very first incarnation of bAdkOde was written in Java and I actually posted it (or made a blog entry about it) over 8 years ago. For some reason I took it down. Probably because I stopped working on it. Anyway, I redesigned the language and wrote an interpreter for it in Perl about 4 or 5 years ago. I finally got around to posting it. Check out the project page for more details. Let me know what you think.

An update to the Grinder testing-framework

Mukesh alerted me to a problem with the Perl conversion script (that converts the XML produced by the Grinder recorder into a Jython file). It wasn’t parsing all the parameters in a GET request properly. I’ve fixed the bug and uploaded a new version of the script. You can download it here.

Sulekha: A text-based Markov-chain generator

I’ve finished work on the projects page. Sometimes I’ll have a passing interest in something that leads me to write some code for fun. Yes, you read right. I like to code… for fun. Anyway, this is something I’ve done ever since I started programming way back in 1991. I’ve written a bunch of cool little programs over the years. Most of these have been throwaway scripts that I wrote just to “see what would happen”. A few of these, however, have had some serious effort put into them. I’ve decided to showcase these on my site. Hopefully people will find them interesting :).

The project I’ve just uploaded is called Sulekha, and is a text-based Markov-chain generator. You can check it out here. The comments for this journal entry are mirrored on the Sulekha project page, so anything you post here will be visible there (and vice versa). Let me know what you think. Oh, and it’s interactive so you can generate your own text-based Markov-chains too!

Artificial Life – some preliminary findings

I fnished my artificial life simulator over the weekend, and I have been running simulations on it. There were a few bugs initially that I quickly fixed. My initial simulations gave me some inexplicable “population explosion”. For example, in one cycle I would have 31 creatures and in the next cycle I would suddenly have around 9,000. It didn’t make very much sense until I analyzed the log files.

If any of you remember, I mentioned earlier that my “creatures” are nothing more than assembly programs. Each of these instructions (and parameters) translate into a bit pattern. Changes in the bit patterns lead to changes in the behaviour of the creature.

One of the instructions is a “reproduce instruction”. This prompts the creature to create a copy of itself, with possible mutations. The instruction has a parameter that tells the creature how many copies to create. During the course of the simulation, certain creatures evolved with the reproduce instruction as their first instruction.

My supervisor program (God program) runs one instruction of every creature during one cycle (sort of like an pre-emptive multitasking operating system). If a creature happens to reproduce during the current cycle, then the supervisor program will execute the child’s instructions as well in that current cycle. This led to an interesting situation. When the mutant that had the reproduce instruction as the first instruction reproduced, its child (if it didn’t have a mutation that changed the instruction) also had the reproduce instruction as the first instruction. So the supervisor program would run the child, and the child would reproduce with a similar offspring. This went on until a mutant that did not have a “reproduce instruction” as the first instruction, evolved. As you can see, this is what led to the population explosion. I corrected this problem by adding “sexual maturity” and “reproductive energy threshold” parameters to the creatures. Basically, a creature can reproduce only if it has been in existence for a certain number of cycles, and if its current energy is above its reproductive energy threshold. This got rid of the population explosion problem.

My simulations gave me some other interesting results. One of the instructions that I have is a MOV instruction. This instruction moves a creature from one cell in the two-dimensional array to another cell. There is a variant of this instruction, known as the MOVA instruction. In the MOV instruction, if the creature tries to move to an occupied cell, it will retreat to its original position. However, in the MOVA instruction the creature will try to kill (and subsequently eat) the occupant if the destination cell is occupied. I had created a creature that simply ate, reproduced, and moved to different cells. I noted that after many generations, this creature had evolved into an aggressive one, that would eat, reproduce, and move aggressively (MOVA) into other cells. That was rather interesting!

I also saw natural selection at work. When I corrected my population explosion problem, I made it so that the reproductive threshold of a creature was higher than its initial (starting) energy. Over the course of many cycles, I saw that the creatures evolved such that the initial energy would be very high, and that their reproductive threshold would be very low. In additon, their reproductive age was lowered. I thought that my creatures were having it too easy, so I modified my code to make reproduction a costly instruction. After running my simulation, I noticed the opposite! Now the creatures evolved such that the reproductive energy threshold would be much higher than their initial energy. It would seem that the creatures wanted to build up enough energy to where they would be able to reproduce successfuly.

There is also one very interesting behaviour that I noticed. I provide branch instructions in my instruction set. A creature can branch to any part of its code. In addition, I also have a Decrement and Branch instruction that is useful for iterative loops. One of my creatures evolved a clever strategy. It would decrement and branch into the middle of the branch instruction. When I analyzed the code, I noticed in doing so, the creature was performing a MOVA instruction in fewer bytes than writing an explicit MOVA instruction. It also had the added benefit of performing the MOVA instruction in an iterative fashion so that it could travel around the two-dimensional array. There was a reason for this kind of behaviour though. A creature consumes energy when it performs an instruction, and the energy consumption is proportional to its code size, so it is amazing, but not surprising that such behaviour could arise.

There is still more work that I need to do. I am thinking of giving my creatures some “memory” in the form of stack and RAM. They will have some Store, Load, Push and Pop instructions whereby they can write to and read from memory. Also, I am thinking of adding some more “registers”. Right now, the creatures have two count registers for performing iterative loops. My reason for adding more is this – I want the creatures to be able to act on the data in memory. For example, I can provide instructions that allow the creature to check the status of a particular cell, like if it is occupied, how much food it has, and so on. The creature can load data into the registers, and then use these registers as arguments for the cell checking instructions. This way I am hoping that my creatures will be able to evolve some rudimentary form of “intelligence”, or “intelligent” behavior. This will also increase my instruction set. Right now, if a mutation results in an invalid instruction, the creature dies when it tries to execute it. Actually, that reminds me – I had creatures that evolved with invalid instructions, but they had also evolved branch instructions that would jump over the invalid section of code!

Most of these complex behaviours aren’t that surprising when you consider natural selection… it is only err… natural that such behaviour should arise!

Well I guess that’s all I have for now. I might make a separate page for this project where I can describe this project in more detail. I’ll post more updates here after I run more simulations.

Intel and Artificial Life

This is my fourth day at Intel. I think I will enjoy working here. I haven’t done much yet though, and I still don’t know exactly what it is I am supposed to be doing. I tested some 140 chips yesterday, but that wasn’t too much work. My supervisor is out until Tuesday, so until then I am doing some odd jobs and reading up some documentation about the platforms and tools we use. I am excited to be working here. It looks like it’s going to be hard work, but I think it will also be fun.

Now about the Artificial Life. I have written up some specs on an Artificial Life program. This is different from earlier ones. I am implementing the “creatures” like assembly programs. This code is their “DNA”. It describes how they behave. A “God Program” executes one instruction of each “creature” and moves on to the next. Sort of like a pre-emptive multitasking operating system. The creatures all live in a two-dimensional area of memory. They can reproduce, and there is chance of error during reproduction.

I think I was going about it the wrong way earlier. I was trying to describe behaviour through variables and through discrete quantities. Now, I represent each creature as a bit-pattern – exactly like an assembly program. I have instructions for moving, eating, moving and attacking, reproducing, and sleeping. In addition, I have some very basic branches and loops. So based on the bit pattern, the creature has a certain behaviour. Also, changes in the bit-pattern can cause changes in behaviour. They can be subtle, or drastic. At least, that is what I believe. I need to actually implement my idea before I can find out! My God program is essentially an operating system, and the “creatures” are tasks. There’s a little more to all of this, but I can’t write it all here. If and when I finish the program, I will make a separate page for it and describe it in detail.

I am thinking of calling this program “Eden”. Anyway, that’s all for now!

Whoa

I had my very first code review. It was interesting and slightly unnerving. I had all these people who had gone over my code, tell me what was wrong with it. Luckily, nothing too serious! Just minor issues. I learnt some new stuff though.

THEN, I found out that my highschool friend is GETTING MARRIED! Yeah, he’s getting engaged this year… Crazy stuff… I can’t believe it…

All original content on these pages is fingerprinted and certified by Digiprove
%d bloggers like this: