I get asked a lot about using Orion (which requires SQL as a database backend) with a SAN. This usually comes up when people are also leveraging the Orion NetFlow Traffic Analyzer (NTA) which can cause the database to grow very, very quickly.
Before I get started, let me say that I believe that the product documentation and the official stance of our tech support team is that we don't recommend running Orion w/NTA with a SAN, and for good reason based upon our overall experience in this area. You see, SANs are great for moving and storing very large amounts of data. In many cases you can actually read and write data more quickly to a high-performance SAN than to locally attached disk. The problem is that with applications like Orion you're not moving large chunks of data; instead, you're moving ginormous amounts of itty bitty pieces of data and most SANs just don't have the ability to handle this number of I/O transactions in the timeframes that applications like this demand. Time and time again we've seen issues where data is getting dropped when trying to write to a high-performance SAN but after moving the data to even a moderately performing local disk array the problem goes away.
For example, I worked with a customer recently that was seeing holes within some of the data sets the he was collecting and was leveraging a SAN to house his SQL database. Additionally, when trying to query the database for these results the queries would sometime time out. We turned on some perfmon counters on the SQL server and we were seeing disk queue lengths (read and write) of 200-300. Microsoft recommends that for SQL Servers with high amounts of I/O the disk queue lengths not exceed twice the number of physical disks (which in this case was 13 if I remember correctly). After moving the database to a local disk array (RAID 1+0), the problems went way...
What inspired me to write this post is that last week while I was at InterOP I had a chance to meet with several of the SAN vendors and to review some of their newer technology and it seems like maybe SANs have now evolved to a place where they could be used very effectively in these scenarios and may even out perform local high-speed arrays. I'll have to wait to see, but it definitely seemed promising.
If any of you out there are effectively utilizing SANs in environments please drop a comment with some specifics.
Flame on...
Josh