As historical data is added to an SQL database, the database adds more rows to tables. As the table within the database continues to grow, with more and more rows, it needs more memory space and disk space to load and search for data, resulting in slower and slower performance. When an SQL database gets too slow, the DBA needs to roll-off some older data. The data is either abandoned, “rolled up” to lower resolution (losing data), or placed off-line, (no longer easily accessible).
I think it is worth noting however, there are two sides to the performance issue, “Reading” and “Writing”; and it goes back to the underlying design of how the data is stored. Some designs can write data quickly, but when they read data, the performance is terrible. The design can be optimized for data retrieval, but then writing the data is more difficult and much slower limiting the number of tags and amount of data that a single server can handle.
I know this is a lengthy reply to what seems like a relatively simple question, but it’s really quite a complicated issue.
President and Founder
Have a question you would like me to answer? Email firstname.lastname@example.org