Lecture Notes in Computer Science:Authors’ Instructions for the Preparationof Camera-Ready Contributionsto LNCS/LNAI/LNBI ProceedingsAyush Hofmann1, PaviBarth1, Ursala Beyer1, Christine Günther1, Frank Holzwarth1, Anna Kramer1, and Erika Siebert-Cole1 1 Springer-Verlag, Computer Science Editorial,Tiergartenstr. 17,69121 Heidelberg, Germany{Alfred.Hofmann,Ingrid.Beyer, Christine.
Guenther, Frank.Holzwarth,Anna.Kramer,Erika.Siebert-Cole, LNCS}@Springer.comAbstract. Algorithms and applications for extensive systems have generallyassumed a fairly simplistic failure model: The computer is defined as areliable digital machine, with consistent execution times and meager amount of failures.If failure occurs, recovery can be handled by checkpoint-restart.
With the pushtoward exascale computing, the concern of preserving the reliable, digitalmachine model will become too costly has become even greater, and we must focustowards Resilient programming to improve algorithms. In this Paper we discuss aStrategy for programming resilience in MongoDB.Keywords: Resilient, MongoDB,Idempotent, Algorithm, Queries, Non-Idempotent, 1 IntroductionErrors and randomcorruptions may affect the result computation of many modern machines.
Resilientalgorithms and programming is a strategy that deals with network errors andoutages and command errors. These are algorithms that are designed to performin the presence of memory errors. Mongo DB is a leading and oneof the most prominent open-source document database and a NoSQL database andthis database is written in C++.
The fact that Mongo DB stores data in JSON-likedocuments that can vary in structure and sizes. Related information is storedtogether for fast query access through the MongoDB query language. MongoDB usesdynamic schemas, meaning that you can create records without first defining thestructure, such as the fields or the types of their values. You can change thestructure of records (which we call documents) simply by adding new fields ordeleting existing ones. This data model, helps you with the ability torepresent stratified relationships, to store arrays, and other more complexstructures easily.
Documents in a compilation need not have a similar set offields and denormalization of data is quite common. Mongo DB, was also designedwith high availability and scalability in consciousness, and includesout-of-the-box replication and auto-sharding. Example of insert, find andupdate queries in MongoDB. 1.1 Code db.users.
insert({ user_id: ‘bcd001’, age: 45, status: ‘A’}) db.users.find() db.users.update( { age: { $gt: 25 } }, { $set: { status: ‘C’ } }, { multi: true }).2.Literature SurveyThissection of the paper surveys many research papers that what are the problemsthat Mongo DB faces and how resilient data structure can help for it.
1proposes the resilient SDN (Software Defined Network) based architecture forIndustrial Control Networks and it portrays several SDN based fasttechnologies. In 2 a design theory isdeveloped for resilient software intensive-systems because of which differentcommunities having different technologies can share a common frame of referenceand knowledge. In 3 Persistency-of-communication (PoC) is been introducedthat deals with the communication failure problem of self triggered consensusnetwork. 4 Provides jest of resilient software defined networks fortechnology related disasters. In 5 focus is mainly on the automatic loading andbalancing of the Mongo DB and proposes heat based load balancing which is very economical.In 6 modeling style style of Mongo DB or NoSQL is discussed that ifnormalization is extended as it has less space for document or collection, thenhow it reduces the query execution time without using join operations.
7Provides the modeling of the Mongo DB with the relational algebra and alsoanalyses the feasibility of moving from SQL database to NoSQL. It does providethe optimization of the Mongo DB. 8 Proposes a method which can achieverobustness to transmission errors of Dirac which is an open source video codec.9 Proposes an algorithm that detects fault tolerant termination based ofDijkstra that is previous fault sensitive scheme. 10 Proposes a monitor thatcan integrate any Mongo DB deployment with the help of easy configurationchanges results shows the it requires low overhead access. 11 Shows that adocument based on NoSQL (Mongo DB) which uses JSON data and how it explores themerits of NoSQL Databases.
3 Proposed WorkThis Paper will give you deep understanding onMongoDB concepts needed to deploy a resilient database.In orderto develop effective re-salient algorithms and applications, we ?rst needprogramming modelsthat permit us to reason about failure and implement recovery. We’re going to do Update One resiliently:updateOne({‘_id’:’2016-06-28′}, {‘$inc’: {‘counter’: 1}}, upsert=True)The operation counts the number of times an eventoccurred, by incrementing a field named “counter” in a document whoseid is today’s date.
Pass “upsert=True” to create the document if thisis the first event in the day. We Identify three errors that could occur whilesending our message to MongoDB.3.1 Transient ErrorsWhen we send his update One message to Mongo DB,the driver may see a transient error from the network layer, such as a TCPreset or a timeout.
The driver cannot tell if the server received the messageor not, so we don’t know whether his counter was incremented. There are othertransient errors that look the same as a network blip. If the primary servergoes down, the driver gets a network error the next time it tries to send it amessage. This error is brief, since the replica set elects a new primary in acouple seconds. Similarly, if the primary server steps down (it’s stillfunctioning but it has resigned its primary role) it closes all connections.The next time the driver sends a message to the server it thought was primary,it gets a network error or a “not master” reply from the server.
Persistent errorsThere might also be a lasting network outage.When the driver first detects this problem it looks like a blip: the driversends a message and can’t read the response. Again, we cannot tell whether theserver received the message and incremented the counter or not. 3.
2 Command errorsWhen the driver sends a message, MongoDB mightreturn a specific error, saying that the command was received but it could notbe executed. Perhaps the command was misformatted,the server is out of disk space, or Ian’s application isn’t authorized.Handling the ErrorsThe Server Discovery and Monitoring Spec requiresa MongoDB driver to track the state of each server it’s connected to. This datastructure is called the “topology description”. If there’s a networkerror while talking to a server, the driver sets that server’s type to”Unknown”, then throws an exception. The topology description is:Server 1: UnknownServer 2: SecondaryServer 3: SecondaryIn the case of a command error, what the driverthinks about the server hasn’t changed: if the server was a primary or asecondary, it still is. Thus the driver does not change the state of the serverin its topology description, it just throws the exception. Code that is about resilience tries 5 times oreven 10 times.
i = 0while True: try: do_operation() break except network error: i += 1 if i == MAX_RETRY_COUNT: throw In the case of a network blip, we no longer risksundercounting. Now we risk over counting, because if the server read our first updateOne message before we got a network error, then the second update One messageincrements the counter a second time.During a persistent outage, retrying more thanonce wastes time. After the first network error, the driver marks the primaryserver “unknown”; when we retry the operation, it blocks while thedriver attempts to reconnect, checking twice per second for 30 seconds. If allthat effort within the driver code hasn’t succeeded, then trying again from hiscode, reentering the driver’s retry loop, is fruitless.3 LNCSOnlineThe online version of the volume will beavailable in LNCS Online. Members of institutes subscribing to the LectureNotes in Computer Science series have access to all the pdfs of all the onlinepublications. Non-subscribers can only read as far as the abstracts.
If theytry to go beyond this point, they are automatically asked, whether they would liketo order the pdf, and are given instructions as to how to do so.4 ResultsIdempotent operations are those which have thesame outcome whether you do them once or multiple times. If we make all hisoperations idempotent, we can safely retry them without danger of over countingor any other kind of incorrect data from sending the message twice. MongoDB has four kinds of operations: find,insert, delete, and update. Find Queries are naturally idempotent.try: doc = findOne()except network err: doc = findOne()InsertThis insert is idempotent. The only warning is,we have assumed we have no unique index on the collection besides the one on_id that MongoDB automatically creates.
doc = {_id: ObjectId(),…}try: insertOne(doc)except network err: try: insertOne(doc) except DuplicateKeyError: pass # first try worked throw Delete If we delete one document using a unique valuefor the key, then doing it twice is just the same as doing it once.try: deleteOne({‘key’: uniqueValue})except network err: deleteOne({‘key’: uniqueValue}) Update This Update is Naturallyindempotent.
updateOne({ ‘_id’: ‘2016-06-28’},{‘$set’:{‘sunny’: True}}, upsert=True) Dealing with the original non-idempotent updateOne: updateOne({ ‘_id’:’2016-06-28′}, {‘$inc’: {‘counter’: 1}}, upsert=True) We’re going to split it into two parts. Each willbe idempotent, and by transforming this into a pair of idempotent operationswe’ll make it safe to retry. In Part One, we leave N alone, we just add atoken to a “pending” array. we need something unique to go here; an ObjectID does nicely:oid = ObjectId()try: updateOne({ ‘_id’: ‘2016-06-28’}, {‘$addToSet’: {‘pending’: oid}}, upsert=True)except network err: try again, then throw For Part Two, with a single message we query forthe document by its _id and its pending token, delete the pending token, andincrement the counter.try: # Search for the document by _id and pendingtoken. updateOne({‘_id’: ‘2016-06-28’, ‘pending’: oid}, {‘$pull’: {‘pending’: oid}, ‘$inc’: {‘counter’: 1}}, upsert=False)except network err: try again, then throw So we can safely retry this update One. Whetherit’s executed once or twice, the document ends up the same:{ _id: ‘2016-06-28’, counter: N + 1, pending: } 5 LectureNotes in Computer Science in the ISI SCI Expanded. Thealgorithm introduced in this research paper, is a modified programme thatprovides a storage effective alternative for Resilient Programming.
wepresented the algorithm for using a richard tree. it does not use repeatitions, which makes itmore effective over a for loop checker. This superiority increasesexponentially as the lenght of the code increases. In this paper we took quriesand converted them into the proposed Resilient Quries. Manually Coding forResilience results in a zero Energy Model, which is desirable. References1. J.
Vestin, A. Kassler and J. Akerberg,”Resilient software defined networking for industrial controlnetworks,” 2015 10th International Conference on Information,Communications and Signal Processing (ICICS), Singapore, 2015, pp. 1-5.
2. J. Rajamäki and R. Pirinen,”Critical Infrastructure Protection: Towards a Design Theory for ResilientSoftware-Intensive Systems,” 2015 European Intelligence and SecurityInformatics Conference, Manchester, 2015, pp.
184-184. 3. D. Senejohnny, P. Tesi and C. De Persis,”A Jamming-resilient Algorithm for Self-triggered NetworkCoordination,” in IEEE Transactions on Control of Network Systems,vol. PP, no.
99, pp. 1-1. 4. C. Mas Machuca et al.
,”Technology-related disasters: A survey towards disaster-resilientSoftware Defined Networks,” 2016 8th International Workshop onResilient Networks Design and Modeling (RNDM), Halmstad, 2016, pp. 35-42. 5. X.
Wang, H. Chen and Z. Wang,”Research on Improvement of Dynamic Load Balancing inMongoDB,” 2013 IEEE 11th International Conference on Dependable,Autonomic and Secure Computing, Chengdu, 2013, pp. 124-130. 6. A.
Kanade, A. Gopal and S. Kanade,”A study of normalization and embedding in MongoDB,” 2014 IEEEInternational Advance Computing Conference (IACC), Gurgaon, 2014, pp. 416-421.
7. G. Zhao, W. Huang, S. Liang and Y.
Tang,”Modeling MongoDB with Relational Model,” 2013 FourthInternational Conference on Emerging Intelligent Data and Web Technologies,Xi’an, 2013, pp. 115-121. 8. Myo Tun and W. A. C. Fernando, “AnError-Resilient Algorithm Based on Partitioning of the Wavelet TransformCoefficients for a DIRAC Video Codec,” Tenth International Conferenceon Information Visualisation (IV’06), London, England, 2006, pp.
615-620. 9. H. Magal, R. Ianconescu and P. Meron,”A robust error resilient video compression algorithm,” MilitaryCommunications Conference, 1994.
MILCOM ’94. Conference Record, 1994 IEEE, FortMonmouth, NJ, 1994, pp. 247-251 vol.
1. 10. Ten-Hwang Lai and Li-Fen Wu, “An(N-1)-resilient algorithm for distributed termination detection,”in IEEE Transactions on Parallel and Distributed Systems, vol. 6, no. 1,pp.
63-78, Jan 1995. 11. L. Vokorokos, M. Uchnár and A. Baláž,”MongoDB scheme analysis,” 2017 IEEE 21st InternationalConference on Intelligent Engineering Systems (INES), Larnaca, 2017, pp.000067-000070.
12. P. Colombo and E. Ferrari,”Enhancing MongoDB with Purpose-Based Access Control,” in IEEETransactions on Dependable and Secure Computing, vol. 14, no. 6, pp. 591-604,Nov.
-Dec. 1 2017. 13. B. Jose and S.
Abraham, “Exploringthe merits of nosql: A study based on mongodb,” 2017 InternationalConference on Networks & Advances in Computational Technologies (NetACT),Thiruvanthapuram, 2017, pp. 266-271.
