What to do about information overload

Computerworld asks end-users, analysts and the NZ Computer Society about storage trends

Dealing with increasing amounts of data is causing headaches for IT, so what are the storage trends, and what are the benefits - as well potential risks - of storing data overseas? Computerworld sought the views of end-users, analysts and the New Zealand Computer Society.

PixelFusion
Web design and development company PixelFusion in Auckland was faced with a storage challenge when working with its client, movie information website flicks.co.nz.

“What really got the site popularity in the first place was how fast the movie trailers streamed for New Zealand users,” says PixelFusion director Craig Boxall.

The reason for this was that the trailers were stored locally rather than having to be downloaded off international servers. Initially storage was fairly simple, trailers were uploaded to a directory on the website and could be watched from there, he says.

But as the popularity of the site grew, PixelFusion had to change the way trailers were stored. Progressively, over the last four years, data has been moved to a co-location storage system.

As international bandwidth in New Zealand is still very expensive, if large numbers of overseas users start viewing trailers from a local server the cost of sending that information to those users is “incredibly huge”, Boxall says. The solution is to store data in two locations, one server in New Zealand and one in the US. The two servers are synchronized using mirroring software app rsync.

“When a visitor comes to the website we track their IP address to see whether they are national or international and serve up a different version of the trailer dependent on where they are from,” Boxall says.

New Zealand users get the locally based trailer, “which makes it nice and fast”, and international users gets the trailer that is stored in the US, which is more or less free because bandwidth is so cheap over there, he says.

“We are at the point now where we need to take storage to the next level and start looking at cloud storage,” Boxall says.

Flicks is planning to go global with sites starting up in Australia and the UK – so there will be a lot more overseas visitors viewing the international trailers.

“We need to find a better way for them to become ‘first-class citizens’,” he says. “Cloud-based storage would allow everybody [to view] the fastest trailer they could get.”

The company is looking at overseas cloud offerings, such as Amazon and Rackspace, he says.


Gartner: Cloudy mix
Analyst firm Gartner is seeing a number of trends in the area of storage, for example managing storage growth; storage efficiency; and how to manage storage in a virtualised environment, says Sydney-based Gartner research vice president Phillip Sargeant.
Big data is another trend, relating to getting value out of big volumes of data. And then there is the whole nature of cloud, he says.

“Certainly storage in the cloud is a talked about topic, but I don’t think that it is necessarily being deployed on a large degree for serious mission-critical applications,” Sargeant says. “Nevertheless, it is something people are talking about a great deal and it’s a trend that probably won’t go away for a while.”

The majority of SMBs and larger enterprises that Sargeant talks to use a mix of disk and tape storage, he says. Typically, disk for day-to-day backup and recovery, and tape for longer term archiving. More often than not, data is stored off-premise but very rarely offshore, he says. Smaller organisation are still using just tape, mainly because of the cost involved of using disks and keeping them secure.

“Some organisations are using managed services, but not a huge number at this point in time,” he says. “Overwhelmingly, organisations [in Australia and New Zealand] are still managing storage in-house.”

Storage in the cloud “provides quite a lot of possibilities in the future” but is still a work in progress, says Sargeant.

“There are still a lot of areas that need to be addressed, such as performance, quality of service and reliability. It’s not mature enough [yet] for your important applications.”

He believes there will be a mix of cloud and non-cloud solutions. Some storage requirements could move into the cloud, such as archiving or some less mission-critical applications, while more important applications could be kept in-house, he says.

“But slowly, over time, as cloud matures, more and more applications will move there,” he says.

However, at the moment, very few Australian and New Zealand organisations that Sargeant has spoken to are happy to store their data – particularly their confidential data – outside the boundaries of their own country.


US location creates anxiety
Data sovereignty has emerged as the single biggest concern for clients outside the US that are looking at adopting public cloud computing, according to another Gartner analyst, research vice president Brian Prentice.