Ask the Expert: Redefining Video Surveillance Storage from Edge to Core

This discussion is now open for questions. We look forward to a lively and informative event.

Best regards,

Roberto

Hi VSS Experts,

You spent a lot of time with customers and EMC partners at ASIS doing live demos. Tell us about it!

• What were to most common customer questions about the new Video Surveillance Solutions launched by EMC?
• What pain points are the new solutions addressing that matter the most to EMC customers?

One of the more common and interesting questions was how/why to use the EMC solutions to enable analytics in line with the security goals of the various folks we spoke to.   The main goal by these security personnel and decision makers was to help them handle unpredictable scenarios and allow them to utilize their surveillance system data to help without going kookoo in the process ;-)

The most compelling discussion was with a stadium customer.  They want to allow their security personnel identify where a particular "threat" has visited to evaluate potential danger during, before, or after an event.  Most video analytics systems are setup to accommodate identifying real time threats by specifying an activity/scenario to look for, which is all predefined.  The issue with these systems is they do not help entirely with the use case defined, which requires identifying a new person to look for across your site. The idea is to reduce the last 2-4 days of video across 100s of video cameras in the stadium and surrounding area to a few minutes of relevant video for 1-2 security analysts/personnel to evaluate and decide there is no threat or perhaps act accordingly on the potential threat.  In the example provided, this might be via pattern matching of a "face/body/hat/shirt color" to the video over the last few days.  We refer to this as post processing and is common in many systems that EMC is deployed outside of surveillance. 

One element of the post processing architecture that is important for such an analytics ready system is to have 1) shared storage of the video data/metadata and 2) standards based protocols for these analytics applications to access data.   EMC refers to this approach as a Data Lake and can reviewed in the generic IT context in the following reference:  Isilon scale-out Data Lake (white paper)

By depicting a system with an Isilon video repository, evidence repository, case management repository, and even distributed sensor repository, the analytics applications could use any file (SMB, NFS, FTP), object (SWIFT, HTTP), or even Analytics (HDFS) protocol to access and process this data to make the security personnel's life and mission easier.  The key here is that the system simplicity also avoids any IO bottlenecks due to silo's of data. 

Hope this is helpful,

Bryan

Thanks Bryan.

During the live Q&A session at the launch there was one concern that kept on coming up, and that is how the migrations from the edge to the core should be tackled.  What are the best mechanisms and preparations and what effects such a transfer may cause to the bandwidth? Can you give some detailed information on these  concerns?  Thanks in advance!

Hello All,

As far as I know , video surveillance system tends to be large in IO Size with a very high percent  of random distribution, (because of VMS) and they are supposed to write much more than read. At least I have analyze storage system of a customer and found out their average IO Size was 120KB and distribution was 98% write & only 2 % read.

So, Why does everyone use NL-SAS (7.2K rpm) for data storing of video surveillance system?  VSS-100 is equipped with NL-SAS as well.

SAS has a higher IOPS, and in an environment with large IO Size, they can gain more bandwidth.

So Is there any technical reasons for choosing NL-SAS in VSS or it is only because of  lower price and higher capacity?

Thanks,

Ali

Video surveillance storage at the Edge is characterized by many attributes/requirements including availability, simplicity, and certifications with VMS. Another important attribute is cost. We've configured the VNX-VSS to reflect as many of these Edge requirments as possible including scalability and price understanding that the VNX-VSS also has innate capabilities to deliver high system bandwidth (which we've tested) of up to 500 MB/sec - combined with the large capacity drives - gives customers a leading "edge" solution to support a few hundred cameras at multiple bit rates. 

The size of video writes for the majority of Windows based VMS vendors comes in 1MB blocks for variety of file sizes.  The other writes present on the workload vary per VMS vendor implementation, but can be audio/metadata file writes.  The Linux based VMS vendors (maybe 10% of market), are all a little different.  Remember to keep an eye on max and min write sizes that the average is being sampled from and you might be able to distinguish writes for the video files versus writes for other parts of the data recorded.

Video surveillance, in my humble opinion has a few workloads to be aware of and you can treat differently if desired.  The most storage centric is video archiving, which typically is highly sequential and nice sized blocks.  The other spectrum of this is the indexing, which is usally on a database, and this is higly transactional and small block sizes.  Then there are foundational OS functions associated with the normalization of the video stream (usually in memory).

What is interesting is I 100% agree with your assessment of video surveillance workload on block based systems, but I disagree that this is true for non-block based systems.  By using a scale-out filesystem, use of pooled resources (disks, memory, CPU, network) the ability to use SATA disks for video archiving workload is well within bounds of handling the workloads by the storage subsystem if it uses a scale-out filesystem.  With EMC Isilon, we are actually able to use our lowest end node types and disks for video surveillance and achieve bandwidths required for the systems with little/no issues even during an entire Node of 36 drives is in failure state (worst case scenario testing).  The issue in the market, actually, is that the memory allocation mechanisms of the software systems needs improvement many times to go above and beyond in the throughputs.  This is because file/object interfaces add latency when compared to SCSI interface due to protocol semantics and the networks.  EMC has worked with quite a few VMS providers on this front and have very good solutions to leverage.

Sorry about diatribe, but it is a fun space!!!

Bryan