[Dspace-general] Video files

[Peter Walgemoed]

Hello John,

I think you have just experienced the 'tip of the iceberg' data explosion many DSpace users are going to see the coming years. I would recommend not compressing the files (lossy) in case you want to use them in the future with the latest (HD) technology. It will be very difficult for DSpace or any archive system to handle such big files and I think it's better that another system optimized for this task handles it under DSpace governance. With video you have the additional need to handle streaming versions on (proprietary) video servers.

I recommend to do the following:
1. Determine a 'master archive' format for the original video material, that meets your current and future needs.
2. Make a simple Unique Identifier generating program that gives this number to the master video file. 
3. If possible try to do a bitstream checksum on the original master file
4. Generate a metadata file with the mpeg and other metadata (like the checksum) and give it the same name as the master video file. For preservation/archive purposes I would advice to never delete this (small) file, but if it changes create a new version.
5. Import the metadata file with Unique ID in DSpace and manage it there as part of the collection and content metadata enrichment process.
6. For production/publishing convert from the master file to a streaming or any other format and let DSpace manage the links/location process.
7. To safe storage costs, store the master video files on tape or DVD near-line or off-line. Be sure to regularly check the integrity of the files (hence the checksum). 

I think we need to develop DSpace the above mentioned way and let storage resource brokers and other IT systems handle the storage distribution and disaster recovery part. 

I look forward to your and the other DSpace community members comments on this idea and how to best implent it.

Best regards,

Peter Walgemoed


  ----- Original Message ----- 
  From: John Murtagh 
  To: dspace-general at mit.edu 
  Sent: Monday, June 26, 2006 11:41 AM
  Subject: [Dspace-general] Video files


  Hello all,

  I've got 15 hours worth of video footage from our Specialist 

  Research Institutes Open Week - however, despite formatting it into MPEG and cutting the footage into 15 minute snippets the files are just too large to be placed onto DSpace.(We are talking 80MB! And no, we don't have a provision for bit streaming!)

  Has anyone got any solutions to this problem?
  Currently, DSpace only "supports" video/mpeg video files but is there a "best" way of compressing files or using a "known" or "unknown" file format on the files we try to upload? Obviously, we want the size reduced without deteriorating the quality of the images.

  Is it best to wait for further upgrades of DSpace in the near future?

  Appreciative of any thoughts/comments/ideas on this one.


  Thanks in advance

  PS

  We're also going to be trying out the Video in DSpace "hack" as presented here: http://66.249.93.104/search?q=cache:adBQyTqRpE4J:dsug2006.uib.no/archive/krejcir.pdf+dirty+trick+dspace&hl=en&ct=clnk&cd=1&client=operaabout 

  wondered if anyone had successfully tested this work-around?

  Thanks

  John Murtagh
  ________________________________________________


  John Murtagh
  Project Manager - Brunel University Research Archive
  Brunel Library
  Kingston Road
  Uxbridge
  UB8 3PH
  Tel. 0189 526 5417
  E-mail: john.murtagh at brunel.ac.uk
  Intranet: http://dspace.brunel.ac.uk/

  _______________________________________________
  Dspace-general mailing list
  Dspace-general at mit.edu
  http://mailman.mit.edu/mailman/listinfo/dspace-general
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.mit.edu/pipermail/dspace-general/attachments/20060626/655c32a7/attachment.htm