i using mongodb elasticsearch application. elasticsearch creates indexes monitioring oplog collection. when both applications running changes collections in mongodb indexed. problem face if reason had delete , recreate index takes ages(2days) indexing complete.
when looking @ size of oplog default it's capacity 40gb , holding around 60million transactions because of creating fresh index taking long time. best way optimize fresh index creation?
is reduce size of oplog holds less number of transactions , still not affect replication or possible create ttl index(which failed on several attempts) on oplog.
i using elasticsearch mongodb using mongodb river https://github.com/richardwilly98/elasticsearch-river-mongodb/.
any overcome above mentioned issues appreciated.
i not elastic search pro question:
what best way optimize fresh index creation?
does apply little use third party fts techs mongodb.
the first thing note if have lot of records there no easy way around unless prepared lose of them.
the oplog isn't idea this, should seek out using custom script using timers in main collection personally, or change table giving single place query new or updated records.
unless filtering oplog specific records, i.e. inserts, pulling out oplog records including deletes, collection operations , database operations. try stripping out unneeded records oplog search, however, creates new problem; oplog has no indexes or index updating.
this means if start read in manner more appropiate use unindexed query on these 60 million records. result in slow(er) performance.
the oplog having no index updating answers 1 of questions:
is possible create ttl index(which failed on several attempts) on oplog.
nope.
as other 1 of questions:
is reduce size of oplog holds less number of transactions
yes, have smaller recovery window of replication , not lose records "fresh" index part of data indexed. unsure, question, if problem or not.
Comments
Post a Comment