Beefy Boxes and Bandwidth Generously Provided by pair Networks
laziness, impatience, and hubris
 
PerlMonks  

[OT] End of hard disks era, new ways of development

by woolfy (Hermit)
on Aug 09, 2004 at 13:41 UTC ( #381221=perlmeditation: print w/ replies, xml ) Need Help??

liz just pointed me the way to an article about terabyte-size memory cards on Slashdot and The Register. Transfer speeds of 120 Mbps and more.

Just recently I had plans to upgrade my 512Mb FlashCard to a card that has 1 Gb or more. Now I have doubts about upgrading so "soon". The same with buying a new hard disk... we just bought a LaCie 400 Gb BigDisk and we were quite happy with it. But it still is a bulky thing.

A lot has been written about what it means to have such memory cards. I am convinced it means the end of the hard disk era. And that means new possibilities and problems for programmers. The memory cards will be much faster than hard disks, moving the weak point in a system with regards to speed (transfer speed, reaction speed) from the storage device to other parts of a system. But memory cards are removable, so programs have to check more often wether the storage device is still there, or wether it is replaced by another device.

Many programs that I have been involved with, had to handle issues as free disk space (traffic logs, email system, database size etc). The problem of free disk space probably will never cease to exist, but with disk space of terabytes, such problems will be less important. I predict that new generations of programmers just never learn to check for available disk space. They will probably just never experience the "disk full" error.

Terabyte size memory cards... wow, the possibilities. Music, video, books, pictures, databases, literature. Even more than what I already have on my silly little 60Gb hard disk.

Comment on [OT] End of hard disks era, new ways of development
Re: [OT] End of hard disks era, new ways of development
by Joost (Canon) on Aug 09, 2004 at 13:55 UTC
      So you just die, no exception?
      ------------ :Wq Not an editor command: Wq
Re: [OT] End of hard disks era, new ways of development
by diotalevi (Canon) on Aug 09, 2004 at 14:07 UTC
    Flash drives tend to have a maximum number of writes per "section." I understand it is around 100,000 right now. That is ok for somemthing that is user-directed (cuz there's no way you're getting anything like that by hand) but as a hard drive... you'd max out after awhile.
      Flash drives tend to have a maximum number of writes per "section." I understand it is around 100,000 right now. That is ok for somemthing that is user-directed (cuz there's no way you're getting anything like that by hand) but as a hard drive... you'd max out after awhile.

      Ah, but if you have very large amounts of very cheap storage - why delete anything? You have a persistant versioned FS instead and never have to worry about accidentally deleting something you shouldn't again :-)

        Yeah... but your file system needs to get updated as you add onto it. Each update to a section is another write.
Re: [OT] End of hard disks era, new ways of development
by adrianh (Chancellor) on Aug 09, 2004 at 14:12 UTC
    liz just pointed me the way to an article about terabyte-size memory cards on Slashdot and The Register.

    Before anybody gets too excited the cards in question can address up to 2Tb (just like some existing Sony Memory Stick types) . Any cards that actually having 2Tb on them are a long way in the future ;-)

    So - just an announcement of yet another memory card type.

    (Poor reporting on Slashdot/Register - nahh :-)

Re: [OT] End of hard disks era, new ways of development
by hardburn (Abbot) on Aug 09, 2004 at 14:30 UTC

    One of the things that make relational databases so efficient is that they are optimized to run on a spinning magnetic disk (like a hard drive). They take time to know just where the head is located physically over the platter and move it in the most efficient manner. This isn't to say that they won't run faster on a solid-state memory device, just that you throw out a lot of hard work if you do.

    I don't think this is a reason not to move to a solid-state storage device, just something to keep in mind. If you can justify getting a TB memory array for your application, go ahead.

    I don't know if there is any real optimization you can do for solid-state devices, or if those optimizations will even make for a significant speed advantage.

    "There is no shame in being self-taught, only in not trying to learn in the first place." -- Atrus, Myst: The Book of D'ni.

      I don't know if there is any real optimization you can do for solid-state devices, or if those optimizations will even make for a significant speed advantage.

      Did you ever study data structures? Have you compared the speed of doing a linear search with a binary tree search? There are optimisations you can do for solid-state devices, it's just like RAM. That's why people spend time trying to work out what the best data structure is. Otherwise, we'd just dump everything in a list and grep for the results we want. As for hardware-specific optimisations, once there is enough hardware around, if people find that it is going too slow, somebody will find a way to do it faster.

Re: [OT] End of hard disks era, new ways of development
by theorbtwo (Prior) on Aug 09, 2004 at 15:30 UTC

    People have been predicting the end of the hard drive for decades now, normaly within a few months. People have been doing similarly for the end to More's Law with relation to CPU speeds for years. People have been predicting revoltions in the density of storage for years.

    I'll pay attention when I see an ad for it from a reputable source, not as something that will be shipping soon, but as something that they are taking orders for. Oh, and go plot me a graph of the size of an average $200 hard drive over time. For extra credit, correct for varitions of the value of the dollar against the yen.


    Warning: Unless otherwise stated, code is untested. Do not use without understanding. Code is posted in the hopes it is useful, but without warranty. All copyrights are relinquished into the public domain unless otherwise stated. I am not an angel. I am capable of error, and err on a fairly regular basis. If I made a mistake, please let me know (such as by replying to this node).

Re: [OT] End of hard disks era, new ways of development
by tilly (Archbishop) on Aug 09, 2004 at 19:14 UTC
    As the saying goes, when you kill your #1 problem, you promote your #2 problem.

    Disk drive space is, for many purposes, effectively a solved problem. Disk drive speed is another story. Moore's law projections suggest that this is likely to become much more true in the future. See this interview with Jim Gray for more perspective. Hard disks won't necessarily die, but people will have to learn to think about them differently.

    Incidentally 120 Mbps is not that fast. A Mbps is one megabit per second. 120 Mbps will transfer 15 megabytes per second. If you really had a 2 terabyte card that you could access at that speed, it would take a day and a half to scan it. By contrast people are now wiring up gigabit ethernet which transfers data between computers about 10 times as fast.

    Of course 2 terabytes is the capacity of the format, not the current cards. The format is meant to anticipate what might be possible someday. Actual cards are not going to get there for a long time.

Re: [OT] End of hard disks era, new ways of development
by bluto (Curate) on Aug 09, 2004 at 19:27 UTC
    Forget about disk drives, I'm still waiting for the death of tape drives. Where I work, we manage over 2 petabytes of data on tape, and the total amount doubles every year. With that much data, tape is much cheaper than disk. Tape was declared dead decades ago, and it may be dead for home users, but it's not quite dead yet...

      I'll chime one in for the death of floppy disks. About a year ago, I got a brand spanking new Promise SATA disk controller, and all the drivers were sent on a floppy disk. The last time I actually tried to use my floppy drive, it managed to short itself out. The power wires connected to it quickly became hot enough to burn through the insulation. So I'm a little wary of hooking up any of the old floppy drives I have laying around, even for SATA. Good thing Promise has a nice online archive of their drivers.

      (I leave a floppy drive in the case, just because I don't have a cover for my 3.5" bay and need something there to keep the dust out. But it's not hooked up.)

      "There is no shame in being self-taught, only in not trying to learn in the first place." -- Atrus, Myst: The Book of D'ni.

      Another thing I don't like about hard disks is that they are made to stay connected to the computer. That's not safe enough for me when used for my backups due to the possibility of viruses and electrical surges. Here's a pro-tape drive article, and here are some of the tape drives I'll be considering for my little business venture. If it doesn't work out, they'll still be suitable for personal backups. Typical tape drives that home users use are cheaper, but I've heard too many stories of corrupted data.
        You could consider a removable harddrive caddy.
        Get disks/caddies and rotate them.
Re: [OT] End of hard disks era, new ways of development
by gmpassos (Priest) on Aug 10, 2004 at 00:12 UTC
    I want to see the end of the CPU Fan! It's sound is just irritant, and for a faster CPU whe always have a Fan even more irritant.

    Well, the HD noise is already fixed now! ;-P

    Graciliano M. P.
    "Creativity is the expression of the liberty".

Re: [OT] End of hard disks era, new ways of development
by woolfy (Hermit) on Aug 10, 2004 at 14:33 UTC
    I was quite surprised by the number of reactions. Thanks for your energy, monks! And of course (almost) all of you were right, but still...

    I know that the end of the tape era end the end of the floppy era have been predicted more times than the end of the hard disk era, and that tapes and floppies still are used. But they are hardly used by consumers nowadays (well, if I can use the advertisements for new computers as a guide for that assumption; ads for tapes and floppies, tape units and floppy drives seem to be a rare commodity now). Within the next few years, I think the same will happen with hard disks. Not within a few months. And as with tapes and floppies, they will be used for many years to come. But as with tapes and floppies, they will fade out of existence. I still remember my first 20 Mb hard disk, it was a removable Tandon disk, bigger than a brick, and I was so proud of it. Hard to imagine now, having USB memory sticks with many more times storage space on them.

    I have read the articles about this 2 Tb memory card a bit better now and indeed, it says "up to" 2 Tb, and it's not yet here. Still, even a "mere" 500 Gb would be wonderful...

    I know 120 Mbps is not that fast, but compared to 2 years ago it certainly is. And I certainly rely on future improvements in that area.

    And I know that memory cards wear out faster than hard disks, so after 100,000 write operations they are less reliable than hard disks. I assume that will be improved in the near future. Let's rely on that a bit than.

    Not checking for available disk space is just a sin. A sin that leads to errors. I know I had those errors quite often myself. My bad. Haven't had that error for quite a while though.

    At the moment, I have hundreds of CD's with backups of my computers, going back to many years. Recently I bought a DVD burner, and that's a big improvement (less disks needed to make backups, less time consuming, less storage space needed). With memory cards of 500 Gb up to 2 Tb, making backups will be so much easier...

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlmeditation [id://381221]
Approved by Elgon
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others romping around the Monastery: (8)
As of 2014-09-01 11:35 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    My favorite cookbook is:










    Results (6 votes), past polls