Menu

📰
0

Reddit - Please wait for verification

learn programming·/u/kneegrow7·3 days ago
#2zWWTt4e
Reading 0:00
15s threshold

I already posted this on r/nestjs but decided to post here. I implemented a scheduler that fetches the entire record except the already processed one and processed it with business logics to and saved it in to another table. Right now, it runs smoothly except it stops at 5500 records out of 100k+ unprocessed records. this.processLogic is where i stored all of the business logics and updates the unprocessedTable isProcessed to true. The Date to skip are those rows that has incomplete data upon ingestion timing (db ingestion on schedule and user has no logout yet or still working at that time) and i have to skip them in order to avoid wrong output. I already finished the project and realised that this module should be fixed as i am expecting thousands of ingestions each day and this should not break. I am thinking about I already created an infinite loop but I dont know where as I am not really a backend guy. I just started in the fullstack ecosystem. Will surely appreciate your suggestions/insights!…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More