Quickly Search for your torrents

As this blog is dedicated to education of people too busy to be too involved in certain niches of technology, and because the blog authors are also too busy to be too involved in any branch of technology (we just want to get the job done with the sharpest axe around) we gotta spend some time explaining the basic technology.

For downloads, there're a lot of developments lately optimized for our huge residential data pipes. For example, I'm on 6.5Mbit/s download and only 384Kbit/s upload. If you sucked at maximum speed all the time, many of the servers you download from would only be able to sustain maximum speed for a small number of people - so they place restrictions on how fast anybody can download at a time, probably to the tune of 800kbit/s - so that other users can also download but at a lower speed.

For truly breathtaking speed, you've gotta download from many people at a time, perhaps hundreds or thousands. This is called peer-to-peer file sharing, but here, we'll discuss the variant called bit-torrent protocol, which enables you to download little pieces of the file you want, from hundreds or thousands at a time, a little at a time, but combined, these little contributions to your download speed could add up to a big big number.

To use the bit-torrent protocol, first you need to download a bit-torrent client. Current `hot choices' are µtorrent, BitComet and azureus all of which have been tested by the community to be `safe' in that they don't contain malware or spyware or viruses. My preference is µtorrent as it is really fast, really small and simple to optimze its connectivity options. On the other end is azureus which is large, filled with options and really flexible with plugins etc. but if you had to use it as a novice, you'd give up this branch of technology because of its daunting configuration options. BitComet has a different encryption scheme from µtorrent and azureus, so you'd probably get better results with µtorrent and azureus. If what I'm saying sounds greek to you, just get µtorrent.

To start downloading using the bit torrent protocol, you first have to find a file with information about the files you want, especially WHERE to get the files from. This information is stored in a .torrent file. Unless you're an out-and-out geek, this is all you need to know.

How to find these .torrent files? Let's say, you want to download the latest and greatest version of ubuntu Linux, what do you do?

You need a search engine. Google is the quintessential search engine, but it may not be updated sufficiently. If you want to use Google, just type: `ubuntu torrent' at the search box, click on the search result and finally the torrent file, and utorrent will automatically start.

A sharper and better way, would be to get to a website which specializes in torrents, like mininova.org but it may not have everything. Even better than that, download a toolbar like Torrent Toolbar (the link is for firefox, but if you search around, IE versions are available) and you get a toolbar with all the major torrent search sites integrated within and also automatically grab RSS feeds from these torrent search engines to inform you on what's new available on their indexes. The problem is that this toolbar doesn't search from all sites at once though, so you have to cycle through the websites to get what you want.

The point of this posting is to announce a wonderful new website - ScrapeTorrent, which integrates all the sites into one search interface and present all the torrents available to you for sorting according to which site has the most seeders, for example, so as to maximize your download speeds. At this time, this is it for me. I'm gonna use this for most of my needs. But I'll still keep the Torrent Toolbar at my firefox just just in case this site goes down.

Face and Image recognition for the masses

Ever wanted to search your entire image archive by typing `Peter' in the search box, and all the pictures with Peter's face in them turn up? Then, enter`Steffi Graf' in the text search box and all her pictures on the web turn up? Or, just upload a picture of a piece of jewelry to a site and it'll search the web for prices of this piece?

Riya.com comes a little closer to this dream. It's an introduction to what google might do with their new acquisition - Neven Vision - a company good at extracting information from photos. Google WAS talking to riya.com before, but I guess it didn't work out.

Discover Magazine has a good writeup on Riya here. Yeah, I still read some nerd print stuff. Coz the authors are more nerdy than some of the web-only types.

Anyway, riya.com just asks you to upload some pictures of your own to their website via a 43MB picture uploader software, and they'll recognize faces, objects, etc. in your photos. Then you can tag them in a number of ways, or riya will autotag them using information from other riya users who appeared in your photos.

Normally, I'd really scream about a site with features as powerful and unique like Riya's. But photos - wow! They're really personal, and people could do bad things with them. To top it all, I've a bad feeling about Riya - check out the bitchiness of Riya's CEO in his personal blog after Google bought Neven Vision instead of Riya. And note the `sour grapes' and incredibly bitchy comments made by the same CEO about Neven Vision and Google. With such bitchiness, I don't know whether I can trust the privacy statements or not. I don't even know whether this company has a future or not. A future in India, maybe, but a global future needs a broader perspective, I think, not the bitchiness demonstrated.

And another stupid, scary thing is, to use one of the modes of autotagging, it requires you to enter your google/yahoo/etc password and username to extract user information and contacts. Come on! That is disrespect of the crudest sense. They could just have an IMPORT CONTACTS function, instead of asking you for one of the most sacred things, your password to your gmail account and of course, your entire google accounts universe, including your credit card number in your checkout account and your personal search information.

So, after they asked for my google username and password, I decided to stop my picture uploading to Riya before it began. I've gotta wait till google's debut of this type of technology. And between google and a bitchy CEO, I'd trust google anytime if I had to.

The bottom line - How much does image recognition sharpen your axe? A whole lot, I think - tagging my 12,000 or more photos, is an IMPOSSIBLE task. With computer-aided tagging, I'd do it. And once the tagging is done, based on facial and object recognition, most of the new pictures would be tagged automatically based on previous object recognition data, unless I've got a new baby or new friends in the pictures. With google's network, perhaps some of your friends have elected to share their own face recognition metrics with every one of their gmail contacts, and I may not even have to manually tag many of my new friends.

If you're a teacher and you'd have to take attendance, just snap away with your point and shoot, 3-4 pictures, and upload it, and there, all 30 of your students identified (unless someone made a funny face, but if he makes funny faces all the time, he'd be recognized too).

And if you found a nice boy and wanted to see how nice this boy really was, use your new K800, take a picture, and see whether this boy appears in some slut's blog as a `good partner'. Or, whether he appears in a picture of a Taliban training camp.

Obviously, with this, nobody can hide, nobody can run. It's pretty much beyond your control - photos usually are taken by friends, and you'd probably have appeared in photos before. Chances are a few of your friends upload the picture, and there - Mr. X identified - in name, with photos.

Doesn't bother me that much. Just hope that Google's CEO isn't as bitchy as Riya's.

When should the axe be sharpened, and how sharp should it be?

Yet another variant of the Woodcutter/Axe story:

A man was walking in a forest one day, and he met a woodcutter. It was a hot day, and he sat down for a smoke and engaged in friendly banter with the woodcutter - things about the weather and such. 5 Marlboros later, he asked, "Mr. Woodcutter, you've been making little progress on chopping down the tree the last half hour. Perhaps your axe is too blunt, why don't you sharpen it?"

"That's the truth! The axe has not been sharpened for a long while now, I could make far better progress with a sharp axe!" said the woodcutter.

"Then why not you take some time to sharpen it now? You'd make good progress!" said the man.

"I don't really have time for that, you know. I gotta chop down all these trees, I don't have the luxury of sharpening my axe," replied the woodcutter.

Everybody I know identifies with this story to some degree, and I see a good number people I know, including myself, in the shoes of the woodcutter at times. What is perfectly clear is this: the axe would eventually have to be sharpened or it will be blunted to such an extent that even minimal progress will be impossible. Questions abound: When do we need to sharpen the axe? How many times do we sharpen? Would a partial sharpening do, or do we need to sharpen it to full keeness?

The principle is clear to me - just sharpen the axe sufficiently, as an when would serve to accomplish your work in the minimum time, with minimal effort expended since effort per unit time is assumed a constant. Draw a crucial distinction between work and effort - work is what you're tasked to do, effort is that which you expend to perform the work.

Abraham Lincoln suggested that 60% of the time budget for chopping down a tree should be spent sharpening it. Well, I did an Excel simulation using the solver tool, and the optimal sharpening/chopping schedule is in the chart below.

Assume that it takes 3 times faster to sharpen an axe than it takes to blunt it in normal chopping. That's a reasonable assumption, I think, considering that a sharpening tool is way more abrasive than wood. Also assume that the chopping power of an axe is directly proportional to the sharpness of the axe, and that the axe gets blunted linearly with respect to time.

Spreadsheet source here.

The areas bounded by the coloured lines and the axes represents the total work done (not effort expended). Both areas are equal. This graph illustrates the time taken to achieve a set quantity of work (in this case, 5000 work units)

The blue line - it takes a woodcutter 100 hours to cut down his quota of trees if he doesn't sharpen his axe. The pink line - it takes the same woodcutter 67 hours to cut down the same quota of trees including a total of 14 hours sharpening his axe, at interspersed intervals, during which no work is done chopping trees.

After 1000 iterations Excel's solver plugin threw at the problem, it recommended to sharpen the axe to its maximum sharpness, as soon as it becomes blunt enough to warrant sharpening. You see those wavy lines? That's when the axe gets sharpened.

I didn't expect this result - in fact I was at a total loss on how the results would be like. I'm not good at mathematics, so I could not get an elegant formula and cancel out as many variables as possible, so I had to resort to a numerical simulation model.

The Excel simulation astounded me. If it's right - we HAVE to keep our tools at a maximum state of performance, and maintain as often as possible. I played with the numbers, and if the axe is allowed to deteriorate further before sharpening than is shown in the chart, you'd have to spend more hours sharpening and thus more time to complete the task. So if you leave an axe to blunt to 70% of its maximum sharpness, and then spend time sharpening it to 100% again, you'd spend more than 67 hours cutting the trees.

Real life is way more complex than this thought exercise - one of the applications of this finding to real life is a deriviative where the `maximum sharpness' is the baseline relative competency when you started work, and `degradation of sharpness' happens when technology changes and new tools come online for other people and you didn't keep up. To work in the most efficient way possible, you'd have to learn every genuine productivity booster which comes online. You can't discount anything.

At least this makes me feel better about my compulsion to try out all the new stuff which comes along which have a chance to shave a few seconds off my work. It's mathematically proven to be the most desired form of behaviour by my employer.

The Second Coming

By Oo Gin Lee

When I first became an IT journalist in 2000, my blood was pumping with adrenaline everyday. It was the height of the dot com boom, and money was a-plenty to fund Jeff Bezos and Jerry Yang wannabes. Heck, all you needed was a decent idea and a .com and watch your shares soar like the wind. Even Li Ka Shing wanted to be part of it, and he saw his shares in tom.com hit the roof. In fact, tom.com had nothing more than a .com in its name and the reputation of Hong Kong's most famous tycoon.

In my team of 6 at the Straits Times Tech & Science Desk, I was initially disappointed that I would not be reviewing the cool gadgets (that went to Steve Dawson, who now is with ESPN) nor the software (that was Francis Chin whose teenage son had a ball of a time with those free games!). Nope, I was tasked by my boss, Tarn How (famous playwright who wrote the screenplay for the first 2 seasons of Growing Up and now with some think tank organisation) to cover the dot commers.

My disappointment soon turned to elation as I began to see the importance of my role. I would make the dot commers famous. I would be their bridge to the world. Help them become millionaires. There was plenty of potential at that time. Patrick Grove of Catcha.com was sizzling hot with his local search engine. The team of cool and good looking dudes and gals from MTV left to start their music website SoundBuzz.com. Then there was earth9.com who wanted to create Habitats, much like what eCircles used to do. How about iteru.net, the guys who started sending SMS messages to huge screens at parties, way before StarHub even thought about Hub TV. Whoever said Singaporeans were uncreative need only look to the dot com era.

In those days, these millionaire wannabes were called Technopreuners and getting Incubated was the hip thing to do. (Incubation means getting funded and getting cheap office space so that your business could grow to a level where you are ready to hatch out of your protective shell) Dot commers were partying every night, meeting venture capitalists and networking with the whole world, including those from the brick-and-mortar traditionals who wanted to get a piece of the money-spinning action. Powering these dot com dreams was a local company called Edge Matrix, who provided the servers and stuff to host the web sites. You may not have heard of the company, but its founder and CEO Dinesh Bhatia hit the news for the wrong reasons years on. Whatever it was, there was plenty of excitement in the air.

I still remember my first break into the Prime pages of The Straits Times. After slogging out for months, I finally met these two good-looking girls who had just set-up an online lingerie store ala Victoria Secretes called efrenchkiss. One was an ex-beauty queen, the other a biz ad graduate. So my first big story, showed these two girls - beauty and brains - holding out a bra at page 4. Their smiles and lingerie filled the page. My words sadly only took up a few lines.

Anyway, as we all know, the dot com fantasy came crashing down one fine day, and with it, the dreams of millions were shattered. Patrick Grove, with his good looks, stopped appearing on the cover of a zillion managazines. He retreated for a while and then made a strong comeback by shifting to the magazine business. Catcha now publishes Juice, Stuff and other magazines. Also alive and kicking, and I suspect doing well, is efrenchkiss, whose website is still selling the sexy lingerie. I remember asking the gals why they sold panties and bras separately instead of a "set" and they shared that in truth, bras were just too expensive so gals had to mix and match! Soundbuzz as you know has made an amazing comeback by actually making people pay to buy music at their portal and signed some great partnerships witht he record labels. As for Edge Matrix and Iteru, their web domains are up for sale right now if you are keen. Earth9 never hit it off with their Habitat dreams but they went to become a leading web solutions company for Microsoft and many other big boys.

As for me. I left The Straits Times in 2002 to set-up my own writing business, mainly to look after my new baby. Thankfully, I still managed to get freelance writing assignments with the papers and IT magazines and the ink is still running in my blood. Unfortunately the demise of the dot com fantasy meant that IT became really boring. Super boring. I turned to reviewing games and gadgets. But now, I can feel my blood pumping again, I can feel the adrenaline flowing again. Now, the excitement is back. And it's called Web 2.0

Simply put, Web 2.0 is the new Web, with super cool applications and tools that you never thought would be possible. Let's start with Google. Back in 2000, Google was this wannabe search engine company who was taking on big guns like Yahoo, Alta Vista, Infoseek, and so on. I remeber telling my suit friends about Google. They poohed-poohed the name. Well, I guess Google had the last laugh. Google's success with online ads meant that it had the muscles and resources to spend some time developing some cool applications. Now with Google software, you can manage your digital picture library, get 2 GB free email, view planet Earth from outer space, earn money by placing ads on your own web-site, search your Outllook e-mail in a jiffy and much much more.

Google was the catalyst. More would follow. Skype, Bit Torrent, Mozilla Firefox are now all household names. Firefox has established itself as serious contender to Internet Explorer, and remains my personal preference for its tabbed browsing and zillion plug-ins.

There are many less well known but equally compelling ones. Jajah.com lets you make phone to phone calls for free. ScanR turns your mobile phone camera into a powerful business card and document scanner. Watch HBO, CNN and other cable TV channels on your PC with TVU

The new Web is here. Once again, I wake up in the morning with a sense of expectation, hoping to discover the next Web application that will blow my mind away. Some of these stuff doesn't even make business sense. It does sound familiar doesn't it? Yes, it's the Second Coming my friends.

Abraham Lincoln and his axe

A vastly underrated quote of Abraham Lincoln's: "If I had six hours to chop down a tree, I'd spend the first four hours sharpening the axe".

Just to state the obvious, the axe is a tool to perform what you have to do. Since the entire human race, including those living in underdeveloped states like Myanmar, has shifted in varying degrees to the information age, I can probably define the axe as `something which helps me do what I have to do faster, better and cheaper'.

I spend an inordinate amount of time sharpening my axe. In my first year at law school, I sharpened my axe till the eve of my exams for Criminal Law, and flunked it. At work, I spend almost all my time sharpening my axe.

But now, with websites like Lifehacker and a ton of like `productivity blogs' around, I find like minded souls, and I spend even more time, if it's possible to sharpen my axe. Trouble is, there seems to be almost no limit to how sharp the axe can be.

So now, I can do a whole lot of things much faster, better and easier than many of my contemporaries. I google faster, I use the spreadsheet faster, I populate the spreadsheet from the database faster. Drawing resources from experts like programmers, and other hackers, and a veritable network of friends and `productivity bloggers', the rate of sharpening is speeding up.

So much so that, in the last 6 months, I feel the gulf between the guys with sharp axes and less keen blades, has widened tremendously. More than it has for the last 3 years. I have no concrete reasons on `why' but I speculate it's because Eric Schmidt, the boss of Google, works harder than Larry Page and Sergey Brin.

Yes, in my mind, Google has driven the industry and provided it with all the catalysts for growth. Google has, by design, taken the stuffy corporate spin out of the web, and replaced it with fun. Witness their blogs - it goes something like:

Well, Google XXX has always prided itself on 2 principles:

1) Our integration of XXX with our great search engine, synergies (blah blah serious corporate stuff) and
2) Our love for jellybeans in Oreo cookies mashed with Ice Cream (fun stuff)

It's their great expertise at the mind game which is screwing Microsoft more than any other. And it's screwing all those suits at Oracle too. Make that, suits everywhere. What the hell do you wear a suit for if a bunch of fun loving kids at Mountain View earn US$800M a quarter and have fun at it? They put the emphasis on STUFFY in stuffy suits.

I'm not here to judge how genuine the fun is at google. What I have to say is that they're doing a damn good job. They put the T-shirts back at Silicon Valley.

Oh and why am I talking about google? Because, the web services which google made so powerful and attractive and fun, has done a helluva lot to sharpen my axe lately. Searches have become better. People are dying to beat google at the game, and labels and tags have given information a whole new dimension. Countless new dimensions. Information quality has improved greatly just because of the new ways to categorize it.

The bottom line: I got this blog to test Blogger Beta, but they didn't allow me to transfer my other blogs to the beta yet, and forced me to register a new name. Since I had this in mind for a long time already, almost 3 months, so I decided to register a new blog.

I don't know whether I have time to fill in this blog past this introduction, but if I do, I hope you get something outta it. I'm gonna show you how I sharpened my axe.