Answer by Amir Afianian for Google scholar blocks python scrapy with Captcha

Try using Google Cache along with a referer.

Also, note not to send more than 2 requests/sec. You may get blocked:

headers = {"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36" ,'referer':'https://www.google.com/'}