Citation:

Sibin Thomas, 2022. "Real-Time Event Processing In Security Camera Systems: An Auto-Sharding and In-Memory Caching Approach Effectively Auto Sharding to Increase Reliability and Cost Efficient ", ESP Journal of Engineering & Technology Advancements, 2(1): 107-111.

Abstract:

This paper addresses the challenge of reliable and cost-efficient event processing at scale, a critical requirement for growing companies. Traditional stateless processing approaches often introduce latency and compromise reliability. We propose a solution based on stateful processing and in-memory caching to overcome these limitations. Our approach leverages stateful processing to ensure reliable event handling and reduce latency. Furthermore, by utilizing in-memory caching, we minimize the cost associated with processing large volumes of events. The paper provides a detailed analysis of our proposed solution, demonstrating its effectiveness in achieving both reliability and cost efficiency for large-scale event processing.

References:

[1] Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep learning. MIT Press.

[2] Lee, K., & Lee, I. (2015). Over-the-air deployment for embedded systems. ACM Transactions on Embedded Computing Systems (TECS), 14 (3), 1-23.

[3] Szegedy, Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D.,... & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9).

[4] Chang, F., Dean, J., Ghemawat, S., Hsieh, W., Wallach, D. A., Burrows, M., & Gruber, R. E. (2006). Bigtable: A distributed storage system for structured data. ACM Transactions on Computer Systems (TOCS), 26 (2), 1-26.

[5] Strauch, C. (2017). streaming data: Understanding the real-time pipeline. O'Reilly Media, Inc.

[6] Kleppmann, M. (2017). Designing data-intensive applications: The big ideas behind reliable, scalable, and maintainable systems. O'Reilly Media, Inc.

[7] Shvachko, K., Kuang, S., Radia, S., & Chansler, R. (2010). the Hadoop Distributed File System. In 2010 IEEE 26th symposium on mass storage systems and technologies (MSST) (pp. 1-10). IEEE.Wang, Zhu, X., Cao, Y., & Liu, J. (2013). Consistent caching with global and local sharding. In Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data (pp. 51-62).

[8] Bates, W., Saria, S., Ohno-Machado, L., Shah, A., & Escobar, G. (2014). Big data in health care: using analytics to identify and manage high-risk and high-cost patients. Health Affairs, 33 (7), 1123-1131.

[9] Siemens & Long, P. (2011). Penetrating the fog: Analytics in learning and educational review, 46(5), 30.

[10] Zheng, Capra, L., Wolfson, O., & Yang, H. (2014). Urban computing: concepts, methodologies, and applications. ACM Transactions on Intelligent Systems and Technology (TIST), 5(3), 1-55.

[11] Alamar, C. (2013). Sports analytics: a guide for coaches, managers, and other decision-makers. Columbia University Press

Keywords:

Security Camera Systems, Event Processing, Auto-Sharding, In-Memory Caching, Real-time Processing, Performance Optimization Cost Efficiency Internet of Things (IoT), Stateful Processing, Optimistic Locking Race Conditions, Latency Reduction.