Saturday, February 22, 2014

Sharing video files of experiments

This post is a summary of responses to a video-sharing inquiry on the SEN Discussion List. Join and participate in Q&A about sediment experiments!

Here are some options for a wide range of video-sharing needs - from small to large files, from private to public sharing, from local hardware to cloud services, and from free to paid… This information might be good to keep in mind when planning or budgeting for new projects. Further comments extremely welcome, especially if you’ve had good or bad experiences with any of these platforms.

Review of the problem:
Large number of large size videos to share privately among a dispersed working group.

The winner: QNAP NAS RAID and Multimedia Library

Pro: LOTS of storage. Easy file sharing. Log-ins. Can share any file format.
Con: Requires some $$ and some set-up.

If I were setting up a lab right now, I’d use some start-up up funds to purchase a QNAP NAS (Network-attached storage). If I were writing a proposal, I’d write it in for data storage. 

I’m far from an expert, but a QNAP 22Tb RAID6 (32Tb raw) just arrived at the Columbia Granular Group. (Read wikipedia’s article about RAID, “redundant array of inexpensive disks”.) In addition to abundant storage for a reasonable price, it comes with extremely easy-to-use firmware for file sharing, a video station, and apps (sync with Google Drive or Dropbox).

With log-in accounts and a logical folder structure that is agreed upon by the group, all group members will be able to access and download what they need. Our humongous raw video files will probably be converted to a standard format like .mp4, then stored in a parallel file structure for browsing. This was the best option for the large volume, easy access, privately-shared video issue that initiated this post. 

Other options:

This is not an exhaustive list, but rather some interesting points I just learned from your recommendations and researching the video-sharing question.

Vimeo

Pro: Several levels of membership for different storage volumes, privacy settings.

Although I had seen videos shared on vimeo before, I wasn’t aware of their membership level details. Memberships are determined by the volume of video per week. Basic (free) is 500 MB per week. Plus ($60 per year) is 5 GB per week, and PRO ($200 per year) is 20 GB per week. Those numbers can add up quickly to give you a large amount of storage per year.

There are no file size restrictions and the videos can be as long as you like. This seems like a good platform for sharing videos to illustrate results (better than just posting the link on a personal website).


Flickr

Pro: Lots of free space, 1 Tb free
Con: Limit of 90 second videos, since it is primarily a photo-sharing site

If you already use Flickr, you may want to take advantage of its generous storage space to share short videos. Flickr video FAQ.


YouTube

Pro: privacy settings, linked to Google (if you like that kind of thing), live streaming (live experiments!)
Con: ads, high probability of being distracted by suggested videos

More academic projects are starting to use YouTube channels as video sharing sites. By clicking a few buttons, you extend the basic account to share videos longer than 15 minutes, and stream live events. There’s integration with a Google account and Google+, if you already have an account set up for your project.

Figshare

Pro: store, share, discover, cite, and on a site specifically for researchers

You may want to check out the figshare post "A YouTube for Scientists". Another option for sharing video (in a citable way!). 

***
Hope to see lots of SEN videos in the future!
Leslie

Monday, January 27, 2014

SEN January 2014 News

Dear Sediment Experimentalists,

The new year and new semester are in full swing. Take a look at some of our new resources and join the discussion listserv so we can start some Q&A about experiment and data management procedures.

Sedimentary Record Articles: SEN’s motivation and early activities are summarized in an article in the recent December 2013 issue of The Sedimentary Record. The issue also has a experimentally-relevant piece on NCED2. Click to read the article...

Visit the new SEN EarthCube Workspace: SEN is funded partially through the NSF EarthCube initiative and we have a SEN page on the EarthCube workspace to show our activities. You can see descriptions of other funded EarthCube projects at workspace.earthcube.org.

We need SEN Knowledge Base Testers: We’re working to get out the first beta version of the Knowledge base and have been testing it with the SEN Fusion Table entries. If you would like to be a beta tester, contact us at sedimentexp@gmail.com.

Join the SEN Listserv: Have a question about experimental procedures or data management and don’t know who to ask? Join our listserv for active discussions on topics for sediment experimentalists. We expect more email traffic on this list than on the newsletter mailing list, so we’ll let you to opt in. The address to post to the list is sedimentexp@earthcube.org.
Register at: http://mail.earthcube.org/mailman/listinfo/sedimentexp_earthcube.org

Friday, December 20, 2013

AGU Report and Holiday Reading

Dear Sediment Experimentalists,

The week in San Francisco at AGU 2013 was busy, and we learned a lot at both the scientific session we co-hosted (link) and the Sediment Experimentalists Network Town Hall. Thanks to ALL participants, presenters, and colleagues who did anything SEN related, you are helping us to build our network!

We leave you at year’s end with a brief Town Hall summary and some holiday reading.
  1. Town Hall highlights: Building a Sediment Experimentalist Network (SEN)
  2. Recommended book: Users Guide to Physical Modelling and Experimentation
  3. Recommended post: The Vast Majority of Raw Data From Old Scientific Studies May Now Be Missing [smithsonian]
  4. Recommended game: Play Delta Shuffle to help scientists understand what controls the shape of deltas

1. Town Hall highlights: Building a Sediment Experimentalist Network (SEN)

On Wednesday, Dec. 11, we described our NSF-funded project for building a Sediment Experimentalist Network, with the three legs of SEN-KB: a Knowledge Base for sharing methods and data, SEN-ED: EDucation and enabling discussions about data and metadata management and guidelines, and SEN-EC: Global Experimental Collaboratories connected through a webinar system for real-time sharing and interaction.

After a demo of the new Knowledge Base (soon launching at sedexp.net!), the conversation hit upon several issues, including:
  • Data storage of multiple TB of data is important and needed
  • Quality of the data shared is a big concern - we should think about peer review of datasets or some other type of quality control
  • There are existing vocabularies from GeoSciML and CSDMS standard names that our community should consider using

2. Users Guide to Physical Modelling and Experimentation: Experience of the HYDRALAB Network (IAHR Design Manual)

The book combines the expertise of many of the leading hydraulic experimentalists in Europe and identifies current best practice for carrying out state-of-the-art, modern laboratory investigations. In addition it gives an inventory and reviews recent advances in instrumentation and equipment that drive present and new developments in the subject. The Guide concentrates on four core areas – waves, breakwaters, sediments and the relatively-new (but rapidly-developing) cross-disciplinary area of hydrodynamics/ecology.

Read more...


3. The Vast Majority of Raw Data From Old Scientific Studies May Now Be Missing (http://blogs.smithsonianmag.com/science/2013/12/the-vast-majority-of-raw-data-from-old-scientific-studies-may-now-be-missing/#ixzz2o2gO5tPU%20)

One of the foundations of the scientific method is the reproducibility of results. In a lab anywhere around the world, a researcher should be able to study the same subject as another scientist and reproduce the same data, or analyze the same data and notice the same patterns.

This is why the findings of a study published today in Current Biology are so concerning. When a group of researchers tried to email the authors of 516 biological studies published between 1991 and 2011 and ask for the raw data, they were dismayed to find that more 90 percent of the oldest data (from papers written more than 20 years ago) were inaccessible. In total, even including papers published as recently as 2011, they were only able to track down the data for 23 percent.

Read more...


4. Play Delta Shuffle to help scientists understand what controls the shape of deltas

Spotted at the AGU NCED Booth:

Calling all Citizen Scientists! Help scientists understand what controls the shape of deltas! PLAY TODAY!

DeltaShuffle.umn.edu

Thursday, October 31, 2013

Sediment Experimentalist Network October News

Dear Sediment Experimentalists,

It’s the end of October and there’s plenty of news for experimentalists.
  1. New video about the Sediment Experimentalist Network (SEN)
  2. New data journal - Nature Scientific Data
  3. Sediment Experimentalists at AGU - Town Hall and Scientific Session

1. New video about the Sediment Experimentalist Network (SEN)

The Sediment Experimentalist Network (SEN) has been funded as part of the NSF EarthCube initiative. Watch this 2-minute video to see what we are planning to do over the next few years, then watch more EarthCube project videos.

 


2. New data journal - Nature Scientific Data

The Nature Publishing group has announced a new journal, Scientific Data, for descriptions of scientifically valuable datasets. Scientific Data “introduces a new type of content called the Data Descriptor designed to make your data more discoverable, interpretable and reusable.”

See their examples of Data Descriptors and think about how to apply it to sedimentary experimental data! [Sample Data Descriptor 1]  [Sample Data Descriptor 2].



3. Sediment Experimentalists at AGU - Town Hall and Scientific Sessions

If you are attending AGU, mark Wednesday December 11th as Sediment Experimentalist Day. We are co-convening scientific sessions and a Town Hall. Hope to see you there!

EP31D and EP32B. Connecting Natural Landscapes to Experimental and Numerical Models of Earth and Planetary Surface Evolution I and II
Wed 8:00 AM - 10:00 AM; 2010 (Moscone West)
Wed 10:20 AM - 12:20 PM; 2010 (Moscone West)

EP33A. Connecting Natural Landscapes to Experimental and Numerical Models of Earth and Planetary Surface Evolution III Posters
Wed 1:40 PM - 6:00 PM; Hall A-C (Moscone South)

Town Hall TH32D. Building a Sediment Experimentalist Network
Wed 12:30 PM - 1:30 PM; 2005 (Moscone West)
We’ll give an update of our activities and share resources and information about best practices experimental methods and data storage, archiving, and dissemination.

--
Sediment Experimentalist Network

Blog: http://sedimentexperiments.blogspot.com/
Website: https://sites.google.com/site/sedimentexperimentalists/
Twitter: http://www.twitter.com/sedimentexp

Sunday, September 15, 2013

Sediment Experimentalist Network September News

Dear Sediment Experimentalists,

1. At the end of August, several of us participated in the Stratodynamics 2013 International Workshop at Nagasaki University, Japan.

2. The SEN Challenge is still open! Help us grow our community presence through online sites and other networking activities. Most tasks only take a minute or two. SEN will award travel support to one of the highest-ranking student participants - all you need to do is complete as many tasks as you can, then email sedimentexp@gmail.com with your finished tasks.

3. For those of you attending the December AGU, keep your eye out for our Town Hall: “Building a Sediment Experimentalist Network”, it will likely be a mid-week lunch spot, but the official schedule will be published in October. We’ll give an update of our activities and share resources and information about best practices experimental methods and data storage, archiving, and dissemination.

Best wishes,
The Sediment Experimentalist Team

Tuesday, August 27, 2013

International Workshop of Stratodynamics at Nagasaki University

Dear Experimentalists,

From August 28-30, we plan to webcast the Stratodynamics Keynote talks from Nagasaki University. Each day starts at 9a local time (8p EDT in the United States)

Aug 28 (Aug 27, 8p in US EDT)
Astrid Blom, Stratigraphy formed by river dunes and Gilbert deltas (45 min)

Aug 29 (Aug 28, 8p in US EDT)
Michael Lamb, What sets the size of river deltas (45 min)

Aug 30 (Aug 29, 8p in US EDT)
Miwa Yokokawa and Norihiro Izumi  (TBD) (90 min)

We will also plan to webcast some of the collaborative experiments, although these will be at 3p local time (2am EDT in the US), so we will try to make some highlight videos to post.

The connection link is as follows:

1. Please join my meeting.
https://global.gotomeeting.com/meeting/join/779601301

2. Use your microphone and speakers (VoIP) - a headset is recommended. Or, call in using your telephone.
United States: +1 (312) 878-3078
Access Code: 779-601-301
Audio PIN: Shown after joining the meeting
Meeting ID:779-601-301


Thursday, August 22, 2013

Sediment Experimentalist Network August News

Dear Experimentalists,

We hope your summer has been a fun and productive one. Here are a few news items before the Fall begins!

1. Take the Sediment Experimentalist Network (SEN) challenge
2. Follow Sediment Experimentalists on Twitter
3. Stratodynamics workshop keynote talks to be broadcast online (August 27, 28, 29 in United States time zones)

1. Take the Sediment Experimentalist Network (SEN) Challenge

Complete as many of the data-related tasks as you can, then email sedimentexp@gmail.com by Sept 1, 2013, with a record of what you completed.
We will choose randomly from the highest-ranking student participants for support to a future SEN activity.

The tasks range from creating Investigator IDs, to short surveys, to joining NSF communities that are talking about data and cyberinfrastructure. Everyone on the mailing list has already completed task #1!

https://sites.google.com/site/sedimentexperimentalists/sen-challenge

2. Follow Sediment Experimentalists on Twitter

The Sediment Experimentalists have joined twitter!

Follow us @SedimentExp  (http://www.twitter.com/sedimentexp)

3. Stratodynamics workshop keynote talks to be broadcast online

The Keynote "Review" talks at next week's Stratodynamics Workshop at Nagasaki University will be broadcast online.

The talks begin at 9:00am on Wednesday, Thursday, and Friday (August 28, 29, 30) local time. (Nagasaki time is GMT/UTC + 09:00 hour, which is US Eastern Daylight Time +13h, so the talks begin at 8:00p on Tuesday, Wednesday, and Thursday for the East coast of the U.S.)

We will post more information here when we have it.

--
Sediment Experimentalist Network

Blog: http://sedimentexperiments.blogspot.com/
Website: https://sites.google.com/site/sedimentexperimentalists/
Twitter: http://www.twitter.com/sedimentexp