Rough Book

random musings of just another computer nerd

Category: Operating Systems

My data-recovery story

I was looking through wayback machine at snapshots of my website, when I came across one from 2005. It reminded me of something I had almost forgotten. At some point in 2005, the network card in my FreeBSD server started to die. I got myself a new card and set about replacing the dying one. I can’t recall why anymore, but I guess I had needed to disconnect the hard-drive at some point. I remember that after I plugged it back in and booted up, I was greeted by a screenful of terrifying error-messages. Something horrible had happened to the drive that held my home directory, my website source-code, and my database. I had lost about 6 years worth of posts and images on my website. My first instinct was to power down the machine to prevent anything more being written to the drive, which I immediately did. After that I think I tried a bunch of disk-recovery tools to try and recover my data. But this was difficult because the filesystem was UFS. I can’t remember if there were any UFS recovery tools at the time, or if I tried them, but I remember having tried almost everything I could think of.

Out of desperation, I think I finally decided to use dd. I started dumping the data from the drive using the lowest size-setting possible in dd (I want to say it is a byte, but I don’t really remember). I then piped this into a perl script that would examine each byte, looking for magic numbers. The drive had been corrupted so badly that there wasn’t even any trace of a coherent filesystem anymore. I knew that the data I was getting were most-probably fragmented, but I didn’t care at this point. I would guess the file-type by looking for magic numbers, and then I would start dumping that data into a file until I found an ending marker, or if the file-type didn’t have one, until the start of another magic number. I remember having various settings in the script so that I could tune its behavior, especially when dealing with false positives. My priority was to retrieve my pictures, website, programming projects, and database. For my source-code I only had to look for ASCII data. For pictures I looked for file markers for JPG, PNG, and GIFs. The database was difficult though, because I was using MySQL. By sheer chance, I had decided to take a SQL dump of my website’s database the day before for backup purposes (ironically, on the very drive that would die the next day). This was ASCII data, and so it was one of the first things my script found.

I ran this script over a couple of hours I think, and then for most of the next day for good measure. Then I began the tedious process of sifting through these files, weeding out false positives. All said and done, I retrieved a good chunk of my data. I think I got back around 80% of my pictures, and almost all of my code and website source. It was a scary few days, but I’m glad that my desperation drove me to try something like this!

How to stop YouTube sucking on Ubuntu/Linux

Recently I’ve noticed that YouTube’s performance on my machines have been terrible. It’s constantly buffering, or it will stop randomly in the middle of a video. I’ll get a few seconds of playback and then 10-30 seconds of buffering. It’s pretty terrible. On Windows I have been able to use the helpful workaround from here and performance has definitely increased. On top of that, I’m also using the SmartVideo plugin on Chrome (it’s also available for FireFox). But on my Linux boxes, I’m still having the same problem in spite of having the SmartVideo plugin. There is a Linux alternative to guide from above, but it uses ipfw program which is not natively available on Ubuntu/Linux (at least from my understanding) due to it being a BSD program. I didn’t want to compile it and install it from source, so I decided to use ufw instead, which is the “Uncomplicated Firewall” that comes with Ubuntu. It was pretty simple to convert the rules over. But first you will need to enable it (if you haven’t already). You can do that with:

sudo ufw enable

Then you can enable logging also, if you want:

sudo ufw logging on

If you SSH into your machine or if you use your machine as a webserver, you will need to enable a few more rules:

sudo ufw allow ssh/tcp
sudo ufw allow http/tcp
sudo ufw allow 8080/tcp

And of course, you can add the rules that will prevent your ISP from caching YouTube:

sudo ufw deny from 173.194.55.0/24
sudo ufw deny from 206.111.0.0/16

You can then use ufw status to verify that your rules are in place:

 ~ ⮀ $ ⮀sudo ufw status
Status: active

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW       Anywhere
80/tcp                     ALLOW       Anywhere
8080/tcp                   ALLOW       Anywhere
Anywhere                   DENY        173.194.55.0/24
Anywhere                   DENY        206.111.0.0/16
22/tcp                     ALLOW       Anywhere (v6)
80/tcp                     ALLOW       Anywhere (v6)
8080/tcp                   ALLOW       Anywhere (v6)

Implementing pinch-zoom and pan/drag in an Android view on the canvas

I was trying to get pinch-zoom and panning working on an Android view today. Basically I was trying to implement the same behavior you see when you use Google Maps (for example). You can zoom in and pan around until the edge of the image, but no further. Also, if the image is fully zoomed out, you can’t pan the image. Implementing the pinch-zoom functionality was pretty easy. I found an example on StackOverflow. I then wanted to implement panning (or dragging) as well. However, I wasn’t able to easily find examples and tutorials for this functionality. I started with this example that comes from the third edition of the Hello, Android! book but I didn’t get too far. So I started playing around a little bit with the events and started writing some code from scratch (using the example from Hello, Android!) so that I could have a better idea of what was happening.

As I mentioned before, getting zoom to work was pretty easy. Implementing panning/dragging was the hard part. The major issues I encountered and subsequently fixed were the following:

  1. Panning continues indefinitely in all directions.
  2. When you zoom and then pan, stop, and then start again, the view jerks to a new position instead of panning from the existing position.
  3. Excessive panning towards the left and top can be constrained, but panning towards the right and bottom is not so easily constrained.

Once I fixed all the problems, I figured that it would be nice to document it for future reference, and I also think it would be a useful resource for others who have the same problem. Now a little disclaimer before I go any further: I’m not an Android expert and I’m really not that great with graphics; I just started it learning to program for Android this semester for one of my Masters electives. So there might be a better way of doing all this, and if there is, please let me know! Also, if you want to skip all the explanations and just see the code, you can skip to the last page.

Read the rest of this entry »

vivin.net is back… mostly

So I’m sure you’ve noticed that this website was down for a while. It went down about three weeks ago when one of the (two) hard-drives on my server died due to bad sectors; it was eight years old. I didn’t panic (too much), because my WordPress database was on the main drive, which is still alive. However, my home directory was on the bad drive and I hadn’t backed up in a while (sigh), and so I lost some images that I had uploaded. What baffled me were my WordPress uploads. I was sure that I had installed WordPress on my main drive, but when I went to search for it, I couldn’t find any trace of the install. Due to this, I’ve lost a few images and I’m not sure if I’ll be able to replace them unfortunately. Oh well.

Also, I must bid farewell to a dear and old friend: my webserver enterprise. I set up my webserver almost 10 years ago (running FreeBSD of course), and she has been serving me faithfully for all this time. Over the years I’ve dealt with all kinds of disasters and I’ve been able to keep her running. However, after this latest disaster I’ve realized that I just don’t have the time to maintain and administer a server anymore; it’s hard to do with a full-time job and with a full-load at school (did I mention that I am doing my Masters?). Therefore, I’ve moved my site over to dreamhost. The cool thing is that they also offer shell access too, which in my opinion is absolutely indispensable. It took me a little while to migrate my WordPress installation over (I had to work out a few kinks), but now it looks like everything is running smoothly.

On that note, I’m looking forward to less server-administration and more blogging!

Rooting the G2

As some of you may already be aware, it appears that the G2 has some sort of “magic restore” (it’s not a rootkit) function that causes all changes to /system to be reverted. This means that you cannot remove any bundled bloatware. Even more troubling, it looks like the phone will perform the restore while it is running (i.e, you don’t need a reset). I haven’t turned on my G2, so this is what I’ve heard from people at XDA Developers. You can get temporary root on the devices, but after a little while (some people say minutes, others say hours; experiences seem to vary) root is lost. So it is possible that something is performing the restore while the phone is running.

Helpful folks on XDA Developers have posted the datasheet to the eMMC and another kind soul (damnoregonian) was able to get the value of the CSD register (the register that seems to control the behavior of the MMC): d00f00320f5903fffffffdff924040c8.

WP_GRP_SIZE[36:32] and WP_GRP_ENABLE[31:31] seem to be the bits that control the write-protect (per the datasheet). Currently these seem to be set to the default values per the data sheet (11111b and 1b). Clearing the bits should (theoretically) turn off the write protection. The value to do that would be d00f00320f5903fffffffde0124040c8. The CSD node is R/O and so you cannot echo to it directly. The only way to do it would be to write a kernel module/driver that writes to the register. Apparently the kernel exports a function called mmc_send_csd, and so one should be able to write to this register.

I’m tempted to write a kernel module that does just that. But since working at Intel I haven’t written any kernel drivers. Also while I did write drivers at Intel, I pretty much made modifications to what others before me had written, and so I never wrote one from scratch. I’m going to see if I can start on something this weekend… wish I was still in college… I had a lot more time then!

If this works (and that’s a big if), there’s still the issue of restores being done while the phone is in operation. That could cause a lot of inconsistency. So this might be a partial solution.

Either way, I’m sure someone will come up with a way to root the phone. But if there’s nothing by next Friday, I’m going to return the phone.

UPDATE

Someone posted updated specs. It looks like those bits are read-only. Bummer. Also, this from T-Mobile’s website:

Bellevue, Wash. — Oct. 7, 2010

As pioneers in Android-powered mobile devices, T-Mobile and HTC strive to support innovation. The T-Mobile G2 is a powerful and highly customizable Android-powered smartphone, which customers can personalize and make their own, from the look of their home screen to adding their favorite applications and more.

The HTC software implementation on the G2 stores some components in read-only memory as a security measure to prevent key operating system software from becoming corrupted and rendering the device inoperable. There is a small subset of highly technical users who may want to modify and re-engineer their devices at the code level, known as “rooting,” but a side effect of HTC’s security measure is that these modifications are temporary and cannot be saved to permanent memory. As a result the original code is restored.

Well, T-Mobile. How about you provide us technical users a way to root our devices? What you’re doing is stupid. You’re going against everything Android stands for. If I can’t root it, I think I will return my phone and get a Vibrant instead. I would have expected this from Apple or Verizon. But not from you. Very disappointing.

Unboxing my new T-Mobile G2 Android Phone

My wife called me at work today and told me that my new phone just came in the mail :). It’s the new G2 by HTC, which is supposed to be the successor to the G1. The specs on this phone are:

  • 3.7 inch WVGA capacitive touch screen
  • 5 megapixel camera with auto-focus and flash
  • Qualcomm MSM7230 800MHz processor
  • 4GB ROM
  • 512MB RAM
  • GPS/aGPS
  • WiFi
  • Bluetooth
  • Android 2.2
  • Runs on T-Mobile’s 4G/HSPA+ network
  • Slide-out keyboard

Read the rest of this entry »

Downtime and System Upgrade

Sorry for the downtime. My webserver had a failing hard-drive and I figured that while I was replacing the hard-drive, I would upgrade the entire machine as well. The box was a Pentium 4 1.4Ghz with 512MB of RAM that I hadn’t upgraded since I first built it in 2002. Now it’s been upgraded to a Pentium 4 2.4Ghz (hyper-threaded) with 1GB of RAM. Building from source will be a lot faster now! The whole upgrade process took a while because I was also in the process of upgrading another one of my machines and that took for EVER (some issues with shorting). Finally I had to install FreeBSD (version 8.0) on the new hard-drive as well as Apache, PHP, MySQL, WordPress etc. After I have everything set up, I’m going to make sure that I image the hard-drive so that I can restore it from backup easily.

Download ShapeWriter APK

I tried to install ShapeWriter from the Android Marketplace yesterday and couldn’t find it (I was trying to reinstall it after flashing my phone with Cyanogenmod 5.0.8). After searching the interwebs, I found out that it had been removed from the marketplace on June 20th indefinitely (supposedly for QA issues). Luckily I had backed up the app using MyBackup Pro and still had the APK, so I was able to reinstall it. I’m putting the APK up here for anyone who needs to reinstall ShapeWriter.

Download ShapeWriter APK

GitHub broke my scp

I set up git on my FreeBSD box so that I can commit my code to GitHub. Today I tried to scp some stuff over and I was met with this rather unhelpful message:

vivin@serenity ~/Projects/code
$ scp -r vivin@www.vivin.net:~/code/agnostic .
Password:
ps: Process environment requires procfs(5)
Initializing new SSH agent...

vivin@serenity ~/Projects/code
$

I fixed the procfs problem by adding the following to my /etc/fstab:

proc                    /proc           procfs  rw              0       0
linproc                 /compat/linux/proc      linprocfs       rw      0       0

and then running:

vivin@enterprise ~
$ sudo mount /compat/linux/proc

vivin@enterprise ~
$ sudo mount /proc  

So I try to scp again and I get:

vivin@serenity ~/Projects/code
$ scp -r vivin@www.vivin.net:~/code/agnostic .
Password:
Initializing new SSH agent...

vivin@serenity ~/Projects/code
$

WTF? Then I remembered making some changes to my .bashrc to be able to commit to github:

function start_agent {
  echo "Initializing new SSH agent..."
  /usr/bin/ssh-agent | sed 's/^echo/#echo/' > "${SSH_ENV}"
  echo succeeded
  chmod 600 "${SSH_ENV}"
  . "${SSH_ENV}" > /dev/null
  /usr/bin/ssh-add;
}

# Source SSH settings, if applicable
if [ -f "${SSH_ENV}" ]; then
  . "${SSH_ENV}" > /dev/null
  #ps ${SSH_AGENT_PID} doesn't work under cywgin
  ps -ef | grep ${SSH_AGENT_PID} | grep ssh-agent$ > /dev/null || {
    start_agent;
  }
else
  start_agent;
fi

I pulled all that out of my .bashrc and made a separate shell script for it. After I did that, scp started working again. I had no idea that calling scp would actually run .bashrc

Trying out ChromeOS from a VMWare image

I was pretty excited when I saw the VMWare image for ChromeOS up for download. I immediately downloaded it to try it out. The zip file I got from gdgt has a vmdk file but no vmx file. I created one from scratch to try ChromeOS out (I’ve made a new zip with the vmdk and the vmx. The link is at the end of this post.). It’s not too bad. The OS boots up really quick. The login screen is pretty spartan (look at the screenshots I have). You login with your Google/Gmail username and password. When you login, it opens up the Chrome browser. I wasn’t able to get anything else running other than the browser. Also, when I first logged in, Chrome complained that the security certificate for Gmail had been revoked. But I was able to login when I typed in the address for Gmail in again. The default tabs seem to be GMail and Google Calendar. I’m assuming that because it’s such an early build, you’re not able to try out the other stuff. Maybe there’s a way; I didn’t play around with it too much. There is a Date and Time settings menu that’s available from the browser, where you can set a few other options, like your proxy, SSL options, home page, and a few other advanced settings.
Read the rest of this entry »

All original content on these pages is fingerprinted and certified by Digiprove
%d bloggers like this: