by Michael Ritacco on January 2, 2009
I was very shocked to see this story today about Journalspace.com.
Journalspace is no more.
DriveSavers called today to inform me that the data was unrecoverable.
Here is what happened: the server which held the journalspace data had two large drives in a RAID configuration. As data is written (such as saving an item to the database), it’s automatically copied to both drives, as a backup mechanism.
There is a big difference between hardware redundancy and logical redundancy. You do not have to have a physical failure to lose your data.
The value of such a setup is that if one drive fails, the server keeps running, using the remaining drive. Since the remaining drive has a copy of the data on the other drive, the data is intact. The administrator simply replaces the drive that’s gone bad, and the server is back to operating with two redundant drives.
But that’s not what happened here. There was no hardware failure. Both drives are operating fine; DriveSavers had no problem in making images of the drives. The data was simply gone. Overwritten.
This sounds like a bad controller, or a bug in the raid controller firmware. As in my cases there was no cause every identified for the failure, and after we reconfigured the RAID 10 array we continued to use the same hardware and never saw the problem again.
The data server had only one purpose: maintaining the journalspace database. There were no other web sites or processes running on the server, and it would be impossible for a software bug in journalspace to overwrite the drives, sector by sector.
The list of potential causes for this disaster is a short one. It includes a catastrophic failure by the operating system (OS X Server, in case you’re interested), or a deliberate effort. A disgruntled member of the Lagomorphics team sabotaged some key servers several months ago after he was caught stealing from the company; as awful as the thought is, we can’t rule out the possibility of additional sabotage.
But, clearly, we failed to take the steps to prevent this from happening. And for that we are very sorry.
So, after nearly six years, journalspace is no more.
I am very sorry for journalspace’s loss, but do appreciate the honesty behind the incident. We can all use this event as a reminder of the importance of backups. Backups are like insurance, you need to pay for what you cannot afford to lose.
As a DBA we must plan for all types of failures, and make sure we explain the differences between physical recovery, logical recovery, high availability, and disaster recovery.
by Michael Ritacco on December 31, 2008
Walk through higher level mathematics on the Dimensions Tour.
This is a very interesting way to visualize the math behind a few of the dimensions the theoretical physicists are always talking about on the Discovery Channel.
It must be all of the the data modeling and star schema development that makes this type of stuff is fascinating to me.
by Michael Ritacco on May 2, 2008
“Technology is cool. It can be incredibly effective way to promote your cause. But hard wires don’t necessarily create human bonds. Your social media strategy can’t simply be a toolset – it needs to be a conduit to living beings. “Java” doesn’t inspire people unless you’re talking about the kind you get from Starbucks. Technology doesn’t compel people. People do.”
“. . . . It is all too easy to fall in love with all the sexy social media tools out there and forget WHY people are attracted to social media in the first place. If you don’t stay grounded in the basic human needs that fuel the success of those shiny tools, you will be – in the words of Nicole Engelbert from Datamonitor – a fool with a tool.”
Great article – a reminder that the engine that drives this and the rest of technology ultimately comes back to people.
by Michael Ritacco on May 1, 2008
Collaboration in the open source community in this HBR article.
Generally, I agree with most of the article, however I think they are leaving out a few critical factors about the type people that make the Linux system work.
The type of people participating is a big factor in terms of any open source solution working. I think ignoring the people factor has been preventing these Collaboration/Knowledge types of organizations from becoming the norm. Since many people must take jobs to survive and not because they are passionate in their work…we keep trying to force something to be that just cannot be sustained in all environments and for any length of time.
The Toyota TPS or Kaizen/Six Sigma/Lean methodologies attempt to make people act in a way (through process) that mirrors the normal operating state for the majority of contributors of open source projects.
Of course incentives play a large part, but only certain types of people will respond to any one type of incentive. I am not sure if the comparison to TPS is fair due to the differences in incentives and the fact that open source participates are self selecting, passionate, and self serving in the results of their work.
What do you think? Have you ever head of the cellular organizational structure?
by Michael Ritacco on April 30, 2008
Experiment with a small group then scale up. Train early adopters to supply leadership and help it catch on. Listen hear
by Michael Ritacco on April 30, 2008
by Michael Ritacco on April 30, 2008
Extensive discussion re: “the heart of social software is the community of users that forms around the software/service.”
I agree with the original post that community and social software are two sides of the same coin. Perhaps we need to identify the community first and then look to what social software is appropriate, rather than the other way around.
Discussion makes clear that social software for technical communication within an enterprise is a different problem than the general case:
“the fluff of social vs. the core of technical enterprise issues.”
“Social software for within the enterprise is low hanging fruit (from the technical perspective). Communication is usually so difficult and fraught within large corporations that improving it is almost trivially easy. And you’ve successfully side stepped the hardest problems of social software on the wider web; identity, investment, community norms all largely come for free. And you get to radically partition your data ala early Facebook! Which makes the scaling challenges of social software disappear.”
“I don’t think using twitter as it exists today is viable in an enterprise setting. Being internally transparent is perceived as good. Using a twitter-like solution outside the firewall would be shot down immediately… Hence the market for social enterprise software.”
by Michael Ritacco on April 30, 2008
Pundit’s gambit on cloud computing:
But, in my opinion, two key factors take cloud computing into a qualitatively different dimension. One is massive scalability. I believe that the kinds of advances that we have become used to in the world of high-end supercomputing are now coming to the more general purpose computing world. I expect that a number of the new applications that data centers will be asked to support will grow by two to three orders of magnitude over the next decade. A 10X – 100X growth over 10 years means roughly that the applications are growing at between 25% and 60% CGR.
Then there is the much higher quality of experience that cloud applications provide to their users. Cloud applications are very different from classic IT applications, whose intrinsic complexities are barely hidden from their users. You truly want users of cloud applications to just be able to access them in the most natural and simplest way possible. Cloud applications should be able to provide a really high quality of experience to massive numbers of users without missing a beat. They should significantly improve the way people deal with the many tasks and devices that surround them in their everyday life – at work, at home, on-the-go, and wherever they happen to be.
by Michael Ritacco on April 30, 2008
by Michael Ritacco on April 28, 2008
Free Beer
It’s ‘free’ in the sense of freedom, not beer give-aways. The organisation is using Creative Commons licences to give public access to its recipe and brand (for pleasure or profit). Usual terms apply: “If you make money selling their unique beer, you have to give them credit and publish any changes you make to the recipe under a similar license”.
All the writing about ‘wisdom of the crowds’ suggests that the Free Beer recipe could be the best one yet!