Dissertation: Enforcing User-Defined Management Logic in Large Scale Systems

As I promised before, following is the information related to my Dissertation and Defence (At Indiana University).

Title: Enforcing User-Defined Management Logic in Large Scale Systems
Year: 2009
Primary Advisor: Prof. Dennis B. Gannon
Thesis Committee: Prof. Dennis B. Gannon (Chair), Prof. Geoffrey C. Fox, , Prof. Beth A. Plale, Prof. David B. Leake, Dr. Sanjiva Weerawarana

Thesis can be found through ProQuest here (I paid and enabled open access, and therefore, you do not need an account to download it). Also there are two papers.

1. Srinath Perera, Dennis Gannon, "Enforcing User-Defined Management Logic in Large Scale Systems," services, pp.243-250, 2009 Congress on Services - I, 2009
2. Srinath Perera, Suresh Marru, Thilina Gunarathne, Dennis Gannon, Beth Plale, "Application of Management Frameworks to Manage Workflow-Based Systems: A Case Study on a Large Scale E-science Project," icws, pp.519-526, 2009 IEEE International Conference on Web Services, 2009

Slides used for the presentation can be found below.

Dissertation Defence: Enforcing User-Defined Management Logic in Large Scale Systems

View more presentations from Srinath Perera.

Abstract:
Due to advances in distributed systems, social motivations, and economic motivations, scales of systems are on the rise. In large-scale systems, changes--caused by failures, maintenance, and additions--are a norm rather than an exception, and therefore, manually keeping these systems running is difficult, if not impossible. System management, which monitors and controls systems, is a prominent solution to this problem.
However, management usecases differ from system to system, yet developing a specific management framework for each system defeats the purpose of building system management frameworks in the first place. Management frameworks that enforce management logic authored by users provide a solution for this problem. These frameworks enable users to change framework's decision logic to cater for user's specific requirements, and after deployed, they monitor and control target systems in accordance to the user-defined management logic. If these logic assert only a single component of the system, we call them local logic, and if these logic assert multiple components in the system, we call them global logic. The global logic depend on a global view about a system, which is non-trivial to support in large-scale systems. However, they enable users to reason about the target system explicitly and, therefore, provide a natural way to express management usecases.
This dissertation presents a new, dynamic, and robust management architecture that manages large-scale systems by enforcing user-defined management logic that depend on a global view of the managed system. Using empirical analysis, we have shown that it scales to manage 100,000 resources, which demonstrates that the architecture can manage most practical systems. This is a testament that despite its dependency on a global view of the managed system, a system management framework can manage systems in accordance to user-defined management logic and can still scale to manage most real world systems. Furthermore, we have demonstrated that the architecture is robust in the face of failures and stable with respect to different operational conditions.

Keynote at CSE Conference

I did a keynote at the CSE conference last Friday. It was a high level talk trying to discuss possibilities (from a Sri Lankan context) in economic hard times. Slides are attached.

CSE Conference Keynote

View more presentations from Srinath Perera.

Computing Research that Changed the World

The video and a List of life Changers

WSO2 Enterprise Service Bus (ESB) 2.1.0 Released!

Details in WSO2 Enterprise Service Bus (ESB) 2.1.0 Released! and Paul Fremantle talking about WSO2 ESB 2.1

Optimizing MPIs for multi-cores : Thoughts by Eran

Nice blogs by Eran, Optimizing MPIs for multi-cores : Thoughts, and also this presentation Functions + Messages + Concurrency = Erlang by Joe Armstrong is pretty good if you need to understand some dynamics of multi-cores and concurrency. In my opinion, Some of his points are open to argument, yet he present a different perspective, obviously helped by his experience with Erlang.

Overhead of Servers on VM's

I was doing a performance tests, and initially trying to run them on virtualized hardware. We ran them first on EC2, then on a Xen based virtualized layer, and finally on real hardware. We observed that the virtualized version is much slower (15 TPS on virtualized vs. 290 TPS on real hardware, almost as 20 times slower!!). Still not sure wheather this applies to servers in general or is this a special case. However, if this is a general result, for servers (which usually has high IO/ CPU overhead) virtualization is really looking bad.

Role of Trust: about "Sri Lanka cricket: an example of what we can do together"

I saw this post by Dr. Sanjiva, Sri Lanka cricket: an example of what we can do together, and I can not agree more. To add my two cents, in my opinion, a part of the problem is that we---Sri Lankans---have lost faith on each other. This faith, the belief that others would not exploit although one has lower his own guard is a great thing, and it always yields a team, which is greater than a sum of it's parts.
One of the example of this trust the history can offer is Phalanx formation practised by Greeks (and later by Swiss) to great effect. One of the key features of the formation is that each man is protected not by his own shield, but by the one hold by the guys in the right, which gave each guy maximum freedom on the battlefield. Then, each man has literally placed his life in the hand of the guy to his right, and trust him to carry out the job. Result was a formidable formation---for instance, Aleksandr the Great used this to its full effect---which was legendry until after decades later Roman's out maneuver them in Battle of Magnesia through the flank (topic of interest, but not relevant to our discussion).
However, it is so easy to preach, and very hard to carry it out, because the trust is mutual. In game theory, Prisoner's dilemma shows how hard (probabilistically) for two untrusting guys to trust each other. For example, with the traffic in the Kandy road, it takes close to 2 hours to drive from Flower road to Gampaha---28km---which says we drive about 14km/hour. I am sure if we all can drive with order, we can get to the destination much faster---I mean how can you drive slower than 15km/h? However, none of us (including me who soon learnt the hard way that no quarter is given there) trust others to not to exploit it if we drive nicely.
So my concluding remark is that trust is such a great thing. Each of us by ourselves can only do so much, and it takes teams to make difference. The trust---or should I say faith, which assures each guy to play his own role, is the key in a great team. But, trust is mutual where it is so hard to keep and easy to break. Every time someone exploit someone else, he contributes to this inertia of untrust by making victim's heart bit more untrustful, and the society (or the team) as a whole takes a step back. You do it over a long time, we get where we are.