Downloading over an unreliable connection with Wget

Rant – BSNL!!!

This is a part rant, part tip – so bear with me… My broadband connection absolutely sucks over the past week. I upgraded from 2Mbps with a download limit to a 4Mbps with unlimited downloads and since then it has been nothing but trouble… Damn BSNL!! I’ve probably registered about 30 odd complaints with them to no avail. If there was a Nobel for bad customer service, BSNL would probably win it by a mile. Some examples:

1. They’ll call to find out what the complaint it and even when I explain what’s happening, they hardly hear me out at all.
2. Either they call up and say ‘We have fixed it at the Exchange’ and nothing has changed
3. They automatically close the complaints

Guess they find it too troublesome that someone who’s paying for broadband actually expects the said broadband connection to work reliably!

Anyway, Airtel doesn’t seem to be any better – they need 10 days to set up a connection and when I was on the phone with them, they didn’t seem too interested in increasing their customer count by 1 .

I also tried calling an ISP called YouBroadband after searching some of the Bangalore forums for good ISP providers. They promised a call in 24 hours to confirm if they have coverage in my area and it was feasible for them to set up the connection and that was 48 hours ago!

At work, I’ve heard good things about ACTBroadband and they have some ads in TOI as well, but they said they don’t have coverage in my area .

So how do you download

Today I needed to download something and doing it from the browser failed each time since my DSL connection would blink out in between!

After ranting and raving and writing the first part above and still mentally screaming at BSNL, decided to do something about it… Time for trusty old wget – surely, it’ll have something?

Turns out that guess was a 100% on the money… it took a few tries experimenting with different options, but finally worked like a charm

wget -t0 --waitretry=5 -c -T5 url
    where
    -t0 - unlimited retries
    --waitretry - seconds to wait between retries
    -c resume partially downloaded files
    -T5 - set all timeouts to 5 seconds. Timeouts here are connect timeout, read timeout and dns timeout

Rewriting history with Git

What’s this about rewriting history?

While developing any significant piece of code, you end up making a lot of incremental advances. Now, it’ll be ideal
if you are able to save your state at each increment with a commit and then proceed forward. This gives you the freedom to try out approaches, go in one way or the other and at each point have a safe harbor to return to. However, this ends up with your history looking messy and folks whom you’re collaborating with have to follow your mental drivel as you slowly built up the feature.
Now imagine if you could do incremental commits but at the same time, before you share your epic with the rest of the world, were able to clean up your history of commits by reordering commits, dropping useless commits, squashing a few commits together (remove those ‘oops missed a change’ commits) and clean up your commit messages and so on and then let it loose on the world!
Git’s interactive rebase lets you do exactly this!!!

git rebase –interactive to the rescue

Git’s magic incantation to rewrite history is git rebase -i. This takes as argument a commit or a branch on which to apply the effects of rewritten rebase operation

Lets see it in operation:

squashing and reordering commits

Let’s say you made two commits A and B. Then you realize that you’ve missed out something which should really have been a part of A, so you fix that with a ‘oops’ commit and call it C. So your history looks like A->B->C whereas you’d like it to look like AC->B

Let’s say your history looks like this:

    bbfd1f6 C                           # ------> HEAD
    94d8c9c B                           # ------> HEAD~1
    5ba6c52 A                           # ------> HEAD~2
    26de234 Some other commit           # ------> HEAD~3
    ....
    ....

You’d like to fix up all commits after ‘some other commit’ – that’s HEAD~3. Fire up git rebase -i HEAD~3

The HEAD~3 needs some explaining – you made 3 commits A, B and C. You’d like to rewrite history on top of the 4th commit before HEAD (HEAD~3). The commit you specify as the base in rebase is not included. Alternatively, you could just pick up the SHA1 for the commit from log and use that in your rebase command.

Git will open your editor with something like this:

    pick 5ba6c52 A
    pick 94d8c9c B
    pick bbfd1f6 C
    # Rebase 7a0ff68..bbfd1f6 onto 7a0ff68
    #
    # Commands:
    #  p, pick = use commit
    #  r, reword = use commit, but edit the commit message
    #  e, edit = use commit, but stop for amending
    #  s, squash = use commit, but meld into previous commit
    #  f, fixup = like "squash", but discard this commit's log message
    #  x, exec = run command (the rest of the line) using shell
    #
    # These lines can be re-ordered; they are executed from top to bottom.
    #
    # If you remove a line here THAT COMMIT WILL BE LOST.
    #
    # However, if you remove everything, the rebase will be aborted.
    #
    # Note that empty commits are commented out

Basically, git is showing you the list of commands it will use to operate on all commits since your starting point. Also, it gives instructions on how to pick (p), squash (s)/fixup (f) or reword(r) each of your commits. To modify the history order, you can simply reorder the lines. If you delete any line altogehter, then that commit totally skipped (However, if you delete all the lines, then the rebase operation is aborted).

So, here we tell that we want to pick A, squash commit C into it and then pick commit B.

    pick 5ba6c52 A
    squash bbfd1f6 C
    pick 94d8c9c B

Save the editor and Git will perform the rebase. It will then pop up another editor window allowing you to give a single commit message for AC (helpfully pre filled with the two original messages for A and C). Once you provide that, git rebase proceeds and now your history looks like AC->B as you’d like it to be.

Miscellaneous tips

Using GitExtensions

1. If you use Git Extensions, you can do the rebase though it’s not very intuitive. First, select the commit on which you’d like the interactive rebase. Right click and choose ‘Rebase on this’.
   
2. This opens the rebase window. In this window, click ‘Show Options’
   
3. In the options, select ‘Interactive rebase’ and hit the ‘Rebase’ button on the right
4. You’ll get an editor window populated similarly.

If the editor window comes up blank then the likely cause is that you have both cygwin and msysgit installed and GitExtensions is using the cygwin version of git. Making sure that msysgit is used in GitExtensions will avoid any such problems.

Using history rewriting

Rewrite history only for what you have not pushed. Modifying history for something that’s shared with others is going to confuse the hell out of them and cause global meltdown. You’ve been warned.

Handling conflicts

You could end up with a conflict – in which case you can simply continue the rebase after resolving the conflicts with a git rebase --continue

Aborting

Sometimes, you just want the parachute to safety in between a rebase. Here, the spell to use is git rebase --abort

Final words

Being able to rewrite history is a admittedly a powerful feature. It might even feel a little esoteric at first glance. However, embracing it gives you the best of both worlds – quick, small commits and a clean history.

Another and probably more important effect is that instead of ‘waiting to get things in shape’ before committing, commits happen all the time. Trying out that ingenious approach that’s still taking shape in your head isn’t a problem now since you always have a point in time to go back to in case things don’t work out.

Being able to work ‘messily’ and commit anytime and being secure in the knowledge that you’d be able fix up stuff later provides an incredible amount of freedom of expression and security. Avoiding the wasted mental cycles spent around planning things carefully before you attack your codebase is worth it’s weight in gold!!!

Compiling Vim again – Cygwin

Vim installed by Cygwin’s setup project does not have Ruby/Python/Perl support enabled by default. As my list of must have vim plugins has a few which use Ruby and Python, thought that it might be good to build my own Cygwin build of Vim. Turned out a little more work than I thought – but that’s more due to the misleading (at least for me ) Make file in the vim source tree called Make_cyg.mak.

Here’s how to compile:

1. Make sure you have python (and ruby, perl and whatever other interpreters you need vim built with) installed.
2. Do not install vim through cygwin (or uninstall it if you have it)
3. Download vim source tarball, untar it and go into the vim73/src folder.
4. Configure

   ./configure --enable-pythoninterp --enable-perlinterp --enable-rubyinterp --enable-gui=no --without-x --enable-multibyte --prefix=/usr
   make && make install
   

5. You’re off to the races!

Creating an interstitial login page with JqueryMobile

So, at work, we’re building a mobile website using JqueryMobile. The app has a bunch of publicly visible pages however, other pages require the user to be authenticated. We didn’t want the user to be forced to login on the first page. Instead, whenever a protected page is accessed, and if the user insn’t logged into the app, we’d like to take him to the login page. Once he’s successfully authenticated, then take him to the page he was navigating to. Doing this in a normal webapp is quite standard – however, with JqueryMobile, query params meddle with the hash navigation model. Also, the page that the user tries to access could be a div in the same physical page or a different url that needs to be fetched.

Trying to solve this was interesting as we were all really just getting started with JqueryMobile – so finding the ideal solution required a few tries. The solution takes a leaf out of JqueryMobile’s approach. The outline of the solution is:

1. Any page div that’s a protected resource is marked with a data-needs-auth="true" attribute
2. We hook into the document level pagebeforechange event to see if the user is trying to transition to a page requiring authentication. If so, then check if we have the user’s authenticated context available.
3. if the said context isnt available,
   1. Cancel default event handling since we’re now going to navigate the user to the login page.
   2. save the toPage object – so once the user is logged in, we know where to take him.
   3. navigate to the login page.
4. In the login page, the page can call the server apis to autheticate the user. Once the user is authenticated, then
   1. See if there’s a valid returnTo object, if so, take the user to the page.
   2. If not, take the user to a ‘default’ page – in our case, this is the app dashboard page.

Code below

    var pageVars = {}
    $(document).bind("pagebeforechange", function (event, data) {
        if (typeof data.toPage == 'object' && data.toPage.attr('data-needs-auth') == 'true') {
            if (!sessionStorage.getItem("TokenSSKey")) {
                if (!localStorage.getItem("TokenLSKey")) {
                    pageVars.returnAfterLogin = data;
                    event.preventDefault();
                    $.mobile.changePage("#Login_Page", { changeHash: false });
                }
                else {
                    sessionStorage.setItem('TokenSSKey', localStorage.getItem("TokenLSKey"));
                }
            }
        }
    });

The login event handler that handles the server response that’s received once we pass the username and password

    function SuccessLogin(data) {
        if (data != null && data.LoginResult != null) {
            if (data.LoginResult.Code === 0) {
                localStorage.setItem('UNameLSKey', data.LoginResult.User.AccountName);
                if ($("#RememberMeChkBx").is(":checked")) {
                    ErrorPanel.html("");
                    localStorage.setItem('TokenLSKey', data.LoginResult.Token);
                    sessionStorage.setItem('TokenSSKey', data.LoginResult.Token);
                }
                else {
                    ErrorPanel.html("");
                    sessionStorage.setItem('TokenSSKey', data.LoginResult.Token);
                }
                if (pageVars && pageVars.returnAfterLogin) {
                    $.mobile.changePage(pageVars.returnAfterLogin.toPage);
                }
                else {
                    $.mobile.changePage("#DashBoard_Page", { changeHash: false });
                }
            }
        }
    }

HOWTO: Access your machine from the internet without a static IP

For machines to be accessible on the internet, usually you need a static IP that’s leased from your ISP so that when someone types in your IP address, so that packets can be routed over to your machine. However, getting a static ip is costly and for the most part, internet users have dynamic IP address that the ISP allocates each time an end user connects to the internet. Since the ip address keeps changing on each connection, there’s no straightforward way to connect to the machine without knowing the IP address that’s been allocated – or so it was at least till Dynamic DNS came along (it isnt new – has been around for ages, but for some reason isn’t that well known)

Typically, when you type in http://www.google.com in your browser, your machine performs a DNS (Domain name service) lookup with the DNS servers from your ISP to find out the IP address corresponding to http://www.google.com. With DDNS (dynamic DNS) this is made to work with your dynamically allocated IP address also. Here’s how it works

1. Register with a DDNS service provider. Service provider provide free accounts for personal use – go to www.dyndns.org
2. Once you’ve created your account, go ahead and set up your hostname. DDNS service providers will have some domains that you can choose from and you get to choose the host part. For a fee, you can also use a domain name of your choice.
3. If your set up has a router at your end, check your router administration page if it supports dynamic DNS. If it does, you need to enter the hostname, account and password. Everytime your router connects to the internet, it sends an update notification to the DDNS service notifying the new IP obtained from your ISP. The DDNS service takes care of sending update notifications to routers on the internet.
4. If you dont have a router, then download the DDNS client software from the service provider. Most DDNS providers have windows, mac and linux clients. These run on your machine and do the same thing – notify the DDNS service provider of your new IP whenever you establish a connection with your ISP.
5. If you’ve got all this set up, then you can reach your machine from the net – try ping <your host name>

If you’re running Linux/Ubuntu, make sure your’re running SSH service and try ssh <your host name>. If you have a router setup, then you will need an additional step – basically the DDNS name refers to your router IP – and not the machine behind the router that you wish to reach. You will also need to make sure that your machine has a static IP from your router. To set up your router, go to your router administration page.

1. Go to the LAN section and give a range of IPs outside of the static IP. Most routers have lan addresses like 192.168.x.y – 192.168.x.z. If you want your host to have an IP address of 192.168.1.100, then give a LAN range that does not include this IP – say 192.168.1.110 – 192.168.1.200.
2. Save and reboot your router.
3. Now go to your network settings and enter your static IP (192.168.1.100), netmask 255.255.255.255, gateway (usually 192.168.1.1).
4. Go to your router administration page and look for a section like virtual server – your router will allow you to forward packets received on a particular port to a host and port within your LAN. You will have to enter the external port (we’ll use 22), the internal machine to forward (192.168.1.100) and the port to forward to (22). With this in place, any packets received on port 22 (ssh) on your router will be forwarded to the 192.168.1.100 machine on the ssh port.
5. Save and reboot your router.
6. Give it a spin.

From a different machine (or from the same one -doesnt matter), try out ssh <your host> and you should be able to login to your machine – via the internet.