WhiteHatBox
Share Page
Following(0)
Partner(s)
user_1234

How do I make my program multi-thread capable?

ReplyThanks 2016/01/14 13:20:08 0 0

I am writing my program and I am trying to figure out how to make it multi-thread compatible.

This is the specific issue I am trying to solve.

How do I create a program that loads 1000 email accounts and does the following:

1. log in once to each account.

2. searches and scrapes data from different websites (no overlapping)

3. logs out of the program and gives a log report and returns variables.


How do I enable multi-threading and make sure the program only goes through 1 account (instead of every thread going through every account.)


Does that make sense? I want to upload one CSV with all account info and run the program with multi-thread enabled while making sure the individual threads don't repeat the function on the same accounts.

greyhat
2016/01/14 15:45:10

My bot making skills are very basic, and so is my knowledge. But this is how I'd probably go about that.


1 Create CSV file and load all accounts into it.


2 Put a "remove row" once an email is used.

This will ensure that your bot does not use the same email twice.


3 Im guessing since you're trying to scrape different sites, then you'd prob need different tables for each each site, or each email account uses its own table for all data that was scraped.


4 Set a global table to save the logs from all accounts (or each account separately)


5) When you run the bot in Runner, you toggle "multi-thread" ON.


Thats the basic of application of what you want. Hope that was helpful enough to you.


Also, another thing to take into consideration is how many threads you'll be trying to run simultaneously.
Since Im using Free version, I can only run 1 thread. My bot was very simple & basic, and the memory Runner was using is about 200mb. So my point is this, if you plan to run 50 threads at a time, 20 times each, Im guessing the memory that you'd need to have would be about 15GB, thats just an estimate compared to my very simple bot.

<< < 1> >>
VerifyCode
Advanced Option