WhiteHatBox
Share Page
Following(0)
Partner(s)
mavillar

BotChiefg always crash!!

ReplyThanks 2017/05/30 05:04:19 0 0

Hi,

I have a module in BotChief. It works fine and the whole module behavies as desired. It consists in a loop scrapping content from a website, and write the content in a file. When I leave it working, in about 15 minutes or so, arises an error saying: BotChief stop working and two options as shown in the image.


Also I compiled the module in an standalone app and the same behavior.

What's happening?

Aprilcaicai
2017/06/01 16:16:45

It occupies so many memories, and also it runs so long. That is why it crashed.


Please do not loop it so long.

mavillar
2017/06/03 18:26:03

Well, I have 64Gb in memory, for working such cases.


All the same, i do not store any information but in local file in each iteration.


Is there any way I could free memory each time the loop start?


I need to leave the loop working for days.

Aprilcaicai
2017/06/05 11:16:21

Sorry sir, you cannot free some memory in each loop start time.


I already forwarded this to our programmers. They will optimize this function if they can figure it out.


Now the solution for it is:

Do not loop it for so long. Add another debugger to continue to scrape.


So sorry for the inconvenience.

mazs91
2019/09/22 12:10:43
any solution currently available to this problem?
Aprilcaicai
2019/09/24 17:18:07

Sorry for the late reply,


Just talked it with the programmer and they told me that the optimization of this one is very difficult and requires more time to study. Please wait for the news.

Dark_1
2019/10/21 20:38:51
Why not scrape the list of pages and visit each page separately in a new thread? This will curb looping?
Karmagedon
2019/10/22 18:21:38

First, try using this option. I have never used it, but maybe it can help.

Rewrite the module so that it exits at the endpoint and the module starts up again. Instead of starting the module once in a cycle, run it as many times as necessary. In this case, the browser will close and restart.

You can try other versions of the browser if this problem did not exist earlier, and it appeared after the latest Chrome updates.

Module Development - wrkgml@gmail.com // Thanks - https://ko-fi.com/arbots
<< < 1> >>
VerifyCode
Advanced Option