Hi,
I have a module in BotChief. It works fine and the whole module behavies as desired. It consists in a loop scrapping content from a website, and write the content in a file. When I leave it working, in about 15 minutes or so, arises an error saying: BotChief stop working and two options as shown in the image.
Also I compiled the module in an standalone app and the same behavior.
What's happening?
It occupies so many memories, and also it runs so long. That is why it crashed.
Please do not loop it so long.
Well, I have 64Gb in memory, for working such cases.
All the same, i do not store any information but in local file in each iteration.
Is there any way I could free memory each time the loop start?
I need to leave the loop working for days.
Sorry sir, you cannot free some memory in each loop start time.
I already forwarded this to our programmers. They will optimize this function if they can figure it out.
Now the solution for it is:
Do not loop it for so long. Add another debugger to continue to scrape.
So sorry for the inconvenience.
Sorry for the late reply,
Just talked it with the programmer and they told me that the optimization of this one is very difficult and requires more time to study. Please wait for the news.
First, try using this option. I have never used it, but maybe it can help.
Rewrite the module so that it exits at the endpoint and the module starts up again. Instead of starting the module once in a cycle, run it as many times as necessary. In this case, the browser will close and restart.
You can try other versions of the browser if this problem did not exist earlier, and it appeared after the latest Chrome updates.
