Quickly Search for your torrents

As this blog is dedicated to education of people too busy to be too involved in certain niches of technology, and because the blog authors are also too busy to be too involved in any branch of technology (we just want to get the job done with the sharpest axe around) we gotta spend some time explaining the basic technology.

For downloads, there're a lot of developments lately optimized for our huge residential data pipes. For example, I'm on 6.5Mbit/s download and only 384Kbit/s upload. If you sucked at maximum speed all the time, many of the servers you download from would only be able to sustain maximum speed for a small number of people - so they place restrictions on how fast anybody can download at a time, probably to the tune of 800kbit/s - so that other users can also download but at a lower speed.

For truly breathtaking speed, you've gotta download from many people at a time, perhaps hundreds or thousands. This is called peer-to-peer file sharing, but here, we'll discuss the variant called bit-torrent protocol, which enables you to download little pieces of the file you want, from hundreds or thousands at a time, a little at a time, but combined, these little contributions to your download speed could add up to a big big number.

To use the bit-torrent protocol, first you need to download a bit-torrent client. Current `hot choices' are µtorrent, BitComet and azureus all of which have been tested by the community to be `safe' in that they don't contain malware or spyware or viruses. My preference is µtorrent as it is really fast, really small and simple to optimze its connectivity options. On the other end is azureus which is large, filled with options and really flexible with plugins etc. but if you had to use it as a novice, you'd give up this branch of technology because of its daunting configuration options. BitComet has a different encryption scheme from µtorrent and azureus, so you'd probably get better results with µtorrent and azureus. If what I'm saying sounds greek to you, just get µtorrent.

To start downloading using the bit torrent protocol, you first have to find a file with information about the files you want, especially WHERE to get the files from. This information is stored in a .torrent file. Unless you're an out-and-out geek, this is all you need to know.

How to find these .torrent files? Let's say, you want to download the latest and greatest version of ubuntu Linux, what do you do?

You need a search engine. Google is the quintessential search engine, but it may not be updated sufficiently. If you want to use Google, just type: `ubuntu torrent' at the search box, click on the search result and finally the torrent file, and utorrent will automatically start.

A sharper and better way, would be to get to a website which specializes in torrents, like mininova.org but it may not have everything. Even better than that, download a toolbar like Torrent Toolbar (the link is for firefox, but if you search around, IE versions are available) and you get a toolbar with all the major torrent search sites integrated within and also automatically grab RSS feeds from these torrent search engines to inform you on what's new available on their indexes. The problem is that this toolbar doesn't search from all sites at once though, so you have to cycle through the websites to get what you want.

The point of this posting is to announce a wonderful new website - ScrapeTorrent, which integrates all the sites into one search interface and present all the torrents available to you for sorting according to which site has the most seeders, for example, so as to maximize your download speeds. At this time, this is it for me. I'm gonna use this for most of my needs. But I'll still keep the Torrent Toolbar at my firefox just just in case this site goes down.

When should the axe be sharpened, and how sharp should it be?

Yet another variant of the Woodcutter/Axe story:

A man was walking in a forest one day, and he met a woodcutter. It was a hot day, and he sat down for a smoke and engaged in friendly banter with the woodcutter - things about the weather and such. 5 Marlboros later, he asked, "Mr. Woodcutter, you've been making little progress on chopping down the tree the last half hour. Perhaps your axe is too blunt, why don't you sharpen it?"

"That's the truth! The axe has not been sharpened for a long while now, I could make far better progress with a sharp axe!" said the woodcutter.

"Then why not you take some time to sharpen it now? You'd make good progress!" said the man.

"I don't really have time for that, you know. I gotta chop down all these trees, I don't have the luxury of sharpening my axe," replied the woodcutter.

Everybody I know identifies with this story to some degree, and I see a good number people I know, including myself, in the shoes of the woodcutter at times. What is perfectly clear is this: the axe would eventually have to be sharpened or it will be blunted to such an extent that even minimal progress will be impossible. Questions abound: When do we need to sharpen the axe? How many times do we sharpen? Would a partial sharpening do, or do we need to sharpen it to full keeness?

The principle is clear to me - just sharpen the axe sufficiently, as an when would serve to accomplish your work in the minimum time, with minimal effort expended since effort per unit time is assumed a constant. Draw a crucial distinction between work and effort - work is what you're tasked to do, effort is that which you expend to perform the work.

Abraham Lincoln suggested that 60% of the time budget for chopping down a tree should be spent sharpening it. Well, I did an Excel simulation using the solver tool, and the optimal sharpening/chopping schedule is in the chart below.

Assume that it takes 3 times faster to sharpen an axe than it takes to blunt it in normal chopping. That's a reasonable assumption, I think, considering that a sharpening tool is way more abrasive than wood. Also assume that the chopping power of an axe is directly proportional to the sharpness of the axe, and that the axe gets blunted linearly with respect to time.

Spreadsheet source here.

The areas bounded by the coloured lines and the axes represents the total work done (not effort expended). Both areas are equal. This graph illustrates the time taken to achieve a set quantity of work (in this case, 5000 work units)

The blue line - it takes a woodcutter 100 hours to cut down his quota of trees if he doesn't sharpen his axe. The pink line - it takes the same woodcutter 67 hours to cut down the same quota of trees including a total of 14 hours sharpening his axe, at interspersed intervals, during which no work is done chopping trees.

After 1000 iterations Excel's solver plugin threw at the problem, it recommended to sharpen the axe to its maximum sharpness, as soon as it becomes blunt enough to warrant sharpening. You see those wavy lines? That's when the axe gets sharpened.

I didn't expect this result - in fact I was at a total loss on how the results would be like. I'm not good at mathematics, so I could not get an elegant formula and cancel out as many variables as possible, so I had to resort to a numerical simulation model.

The Excel simulation astounded me. If it's right - we HAVE to keep our tools at a maximum state of performance, and maintain as often as possible. I played with the numbers, and if the axe is allowed to deteriorate further before sharpening than is shown in the chart, you'd have to spend more hours sharpening and thus more time to complete the task. So if you leave an axe to blunt to 70% of its maximum sharpness, and then spend time sharpening it to 100% again, you'd spend more than 67 hours cutting the trees.

Real life is way more complex than this thought exercise - one of the applications of this finding to real life is a deriviative where the `maximum sharpness' is the baseline relative competency when you started work, and `degradation of sharpness' happens when technology changes and new tools come online for other people and you didn't keep up. To work in the most efficient way possible, you'd have to learn every genuine productivity booster which comes online. You can't discount anything.

At least this makes me feel better about my compulsion to try out all the new stuff which comes along which have a chance to shave a few seconds off my work. It's mathematically proven to be the most desired form of behaviour by my employer.

Abraham Lincoln and his axe

A vastly underrated quote of Abraham Lincoln's: "If I had six hours to chop down a tree, I'd spend the first four hours sharpening the axe".

Just to state the obvious, the axe is a tool to perform what you have to do. Since the entire human race, including those living in underdeveloped states like Myanmar, has shifted in varying degrees to the information age, I can probably define the axe as `something which helps me do what I have to do faster, better and cheaper'.

I spend an inordinate amount of time sharpening my axe. In my first year at law school, I sharpened my axe till the eve of my exams for Criminal Law, and flunked it. At work, I spend almost all my time sharpening my axe.

But now, with websites like Lifehacker and a ton of like `productivity blogs' around, I find like minded souls, and I spend even more time, if it's possible to sharpen my axe. Trouble is, there seems to be almost no limit to how sharp the axe can be.

So now, I can do a whole lot of things much faster, better and easier than many of my contemporaries. I google faster, I use the spreadsheet faster, I populate the spreadsheet from the database faster. Drawing resources from experts like programmers, and other hackers, and a veritable network of friends and `productivity bloggers', the rate of sharpening is speeding up.

So much so that, in the last 6 months, I feel the gulf between the guys with sharp axes and less keen blades, has widened tremendously. More than it has for the last 3 years. I have no concrete reasons on `why' but I speculate it's because Eric Schmidt, the boss of Google, works harder than Larry Page and Sergey Brin.

Yes, in my mind, Google has driven the industry and provided it with all the catalysts for growth. Google has, by design, taken the stuffy corporate spin out of the web, and replaced it with fun. Witness their blogs - it goes something like:

Well, Google XXX has always prided itself on 2 principles:

1) Our integration of XXX with our great search engine, synergies (blah blah serious corporate stuff) and
2) Our love for jellybeans in Oreo cookies mashed with Ice Cream (fun stuff)

It's their great expertise at the mind game which is screwing Microsoft more than any other. And it's screwing all those suits at Oracle too. Make that, suits everywhere. What the hell do you wear a suit for if a bunch of fun loving kids at Mountain View earn US$800M a quarter and have fun at it? They put the emphasis on STUFFY in stuffy suits.

I'm not here to judge how genuine the fun is at google. What I have to say is that they're doing a damn good job. They put the T-shirts back at Silicon Valley.

Oh and why am I talking about google? Because, the web services which google made so powerful and attractive and fun, has done a helluva lot to sharpen my axe lately. Searches have become better. People are dying to beat google at the game, and labels and tags have given information a whole new dimension. Countless new dimensions. Information quality has improved greatly just because of the new ways to categorize it.

The bottom line: I got this blog to test Blogger Beta, but they didn't allow me to transfer my other blogs to the beta yet, and forced me to register a new name. Since I had this in mind for a long time already, almost 3 months, so I decided to register a new blog.

I don't know whether I have time to fill in this blog past this introduction, but if I do, I hope you get something outta it. I'm gonna show you how I sharpened my axe.