Ten years ago, the open source stack of platforms, languages, scripts, and procedures of Python/PHP, MySQL, Apache, and Linux (also known as LAMP) ushered the flowering of low cost start ups. Due to the free infrastructure LAMP provided, startups easily and economically got their web presences up. The low-cost hardware needed for a LAMP set up triggered an avalanche of startups since the barrier to entry in terms of web presence and application development was significantly lowered by this open source stack. Given the huge user base some of these start ups generated, LAMP setups dramatically lowered scaling and upgrade costs. However, there was a major problem – a possibly missed opportunity – that such large user bases using open source infrastructure created.
The problem revolved around data management for each user and transaction intelligence. The opportunity centered on how this mass of users can be used as a collective source of intelligence for user experience upgrades. As startups reached breakout sizes in scale and user population, analyzing data at this large of a scale required precisely the type of infrastructure LAMP was originally designed to sidestep. Expensive software deals and even hardware began to appear as options. A free new open source framework called Hadoop fixes this problem.
Hadoop is a Java-based framework that provides support for distributed applications running on tons of computers processing huge volumes of data. Hadoop lets startup engineers plan for the future while leveraging free resources. No expensive custom software needed – Hadoop is customizable and can be made to fit your network’s ever rising application management and coordination needs. Hadoop definitely takes the spirit of LAMP development one step further in servicing startups’ needs.