ABSTRACT
An efficient and distributed scheme for file mapping or file lookup is critical in decentralizing metadata management within a group of metadata servers. This paper presents a novel technique called Hierarchical Bloom Filter Arrays (HBA) to map filenames to the metadata servers holding their metadata. Two levels of probabilistic arrays, namely, the Bloom filter arrays with different levels of accuracies, are used on each metadata server. One array, with lower accuracy and representing the distribution of the entire metadata, trades accuracy for significantly reduced memory overhead, whereas the other array, with higher accuracy, caches partial distribution information and exploits the temporal locality of file access patterns. Both arrays are replicated to all metadata servers to support fast local lookups. We evaluate HBA through extensive trace-driven simulations and implementation in Linux. Simulation results show our HBA design to be highly effective and efficient in improving the performance and scalability of file systems in clusters with 1,000 to 10,000 nodes (or superclusters) and with the amount of data in the petabyte scale or higher. Our implementation indicates that HBA can reduce the metadata operation time of a single-metadata-server architecture by a factor of up to 43.9 when the system is configured with 16 metadata servers.
TABLE OF CONTENT
TITLE PAGE
CERTIFICATION
APPROVAL
DEDICATION
ACKNOWLEDGEMENT
ABSTRACT
TABLE OF CONTENT
CHAPTER ONE
1.0 INTRODUCTION
1.1 STATEMENT OF PROBLEM
1.2 PURPOSE OF STUDY
1.3 AIMS AND OBJECTIVES
1.4 SCOPE/DELIMITATIONS
1.5 LIMITATIONS/CONSTRAINTS
1.6 DEFINITION OF TERMS
CHAPTER TWO
2.0 LITERATURE REVIEW
CHAPTER THREE
3.0 METHODS FOR FACT FINDING AND DETAILED DISCUSSIONS OF THE SYSTEM
3.1 METHODOLOGIES FOR FACT-FINDING
3.2 DISCUSSIONS
CHAPTER FOUR
4.0 FUTURES, IMPLICATIONS AND CHALLENGES OF THE SYSTEM
4.1 FUTURES
4.2 IMPLICATIONS
4.3 CHALLENGES
CHAPTER FIVE
5.0 RECOMMENDATIONS, SUMMARY AND CONCLUSION
5.1 RECOMMENDATION
5.2 SUMMARY
5.3 CONCLUSION
5.4 REFERENCES
Disclaimer: Note this academic material is intended as a guide for your academic research work. Do not copy word for word. Note: For Computer or Programming related works, some works might not contain source codes
CITE THIS WORK
(2014, 09). Distributed Metadata Management For Large Cluster-based Storage Systems.. ProjectStoc.com. Retrieved 09, 2014, from https://projectstoc.com/read/3138/distributed-metadata-management-for-large-cluster-based-storage-systems-4954
"Distributed Metadata Management For Large Cluster-based Storage Systems." ProjectStoc.com. 09 2014. 2014. 09 2014 <https://projectstoc.com/read/3138/distributed-metadata-management-for-large-cluster-based-storage-systems-4954>.
"Distributed Metadata Management For Large Cluster-based Storage Systems.." ProjectStoc.com. ProjectStoc.com, 09 2014. Web. 09 2014. <https://projectstoc.com/read/3138/distributed-metadata-management-for-large-cluster-based-storage-systems-4954>.
"Distributed Metadata Management For Large Cluster-based Storage Systems.." ProjectStoc.com. 09, 2014. Accessed 09, 2014. https://projectstoc.com/read/3138/distributed-metadata-management-for-large-cluster-based-storage-systems-4954.
- Related Works
- Design And Implementation Of Computer Based Police Investigation System (a Case Study Of Fiib Headquarters Ogbor-hill Aba)
- Design And Implementation Of A Computerized Cargo Transportation Management System (a Case Study Of Super Maritime Shipping Agency, Lagos)
- Design And Implementation Of A Bill Of Materials Processing System (a Case Study Of Emenite Nig. Ltd. Emene, Enugu)
- Application Of Fuzzy Logic In An Elevator
- Mobile Information Provider
- Design And Implementation Of Computerized Pension Scheme Management System (a Case Study Of Nigeria Pension Board Enugu)
- Design And Implementation Of A Computerized Homeopathic Management System (a Case Study Of University Of Nigeria Teaching Hospital Enugu State)
- Design And Implementation Of Computerized Stock Marketing Report (a Case Study Of Nigeria Stock Exchanhe Commission)
- Application Of Mobile Agent In Distributed Multimedia Database System
- Vision Based Processing For Real-time 3d Data Acquisation Based Code Structured Light