WhiteHatBox
Official Twitter Account
Get Support
Create Support Ticket
Share Page
Following(0)
Partner List
mokote

how to scrap data from facebook dynamic pages?

ReplyThanks 2016/04/12 10:44:39 0 0

when you search on facebook you have only few records loaded and to load more you need scroll down, how to do it properly?


regards for whitehatbox team :)

Aprilcaicai
2016/04/12 22:13:03

Hi, mokote

please set the setting like below picture shown

Wish this is clear.


sates@der.pl
2017/10/26 03:36:08

Scrape BodyHTML before Control Operate and after Control Opertae... compare them.
If they are equal, it means that all page data has loaded

Regards :)

Jurek
PercyTapia
2017/12/20 02:09:55
Hi and thanks for the GREAT software.
I am trying to accomplish infinite scroll by comparing body html before and after the scroll in a wait(loop).
It gets to the end of the page but just keeps looping. Is there any other way of doing it apart from using the wait(loop)? Using that loop for a long time makes the software very unresponsive and slow, at least for me.
The scroll takes a very long time doing it this way.
I am trying to scroll to the bottom of this video page: https://www.youtube.com/channel/UCDChM4NHBwzRCs6AOrgFOPA/videos

Thanks very much for any suggestions.

Percy

/editor/attached/file/20171220/20171220020929_3334.zip

PercyTapia
2017/12/27 17:24:16

I managed to get this to work better by adding an extra sleep of 2000ms right after false.

Sometimes it still stays in the loop and doesn't break until after an extra 7 times after the end of the page. A different solution to this would be great.

Aprilcaicai
2017/12/28 16:31:02

We checked this one for you. And i am sorry, currently there is no other way to solve it.


Probably it does not load the page fully, so it loops it again.


The scraped out content is too much, that makes it run very slowly.

PercyTapia
2017/12/31 05:19:23
Thanks for taking a look at this.
It works a lot better and faster in the runner than in the editor.
Percy
Aprilcaicai
2018/01/02 13:25:11

You are welcome.


That is good news.


If there is better solution for it, we will try to optimize it in the future.

<< < 1> >>
VerifyCode
Advanced Option