Puppeteer js not work

  • OS: ubuntu 20.04

  • Programming language and version: Javascript
    Hi everyone, I have a problem with the puppeteer js library:
    On the local environment everything works, but as soon as I do the deploy on Qovery dotn works.
    Can anybody help me, or give me some advice?
    Thank you!

Hello,
can you be a bit more specific ? What’s going wrong ?

1 Like

executing this function the script stops at console.log ((“ASIN =” + m [4])
Everything else does not work, but without errors at runtime

const scraperData = async (url) => {
  const regex = RegExp(
    "https://www.amazon.it/([\\w-]+/)?(dp|gp/product)/(\\w+/)?(\\w{10})"
  );
  m = url.match(regex);

  if (m) {
    console.log("ASIN=" + m[4]);
  }

  const browser = await puppet.launch({ args: ["--no-sandbox"] });
  const page = await browser.newPage();
  await page.goto(url);
  await page.waitForXPath('//*[@id="priceblock_ourprice"]');
  const [priceblock_ourprice] = await page.$x('//*[@id="priceblock_ourprice"]');
  const priceTxt = await priceblock_ourprice.getProperty("textContent");
  let priceFinal = await priceTxt.jsonValue();
  priceFinal = priceFinal.replace("€", "").replace(",", "");

  const [title] = await page.$x('//*[@id="productTitle"]');
  let title_def = await title.getProperty("textContent");
  title_def = await title_def.jsonValue();
  title_def = title_def.trim();
  browser.close();
  return { priceFinal, title_def };
};


but it works in the local environment.

1 Like

Could you give me the URL of your app ?

1 Like

Sorry, where do i find the url in the dashboard?

1 Like

It’s the url in your browser when you’re on you app.

1 Like

ok:

but is an private page…

The url you gave me is linked to a running app and everything looks good. The console.log is well called and return values.

ok but the code after the console.log it doesn’t run, that’s the problem.
below all code…

(async () => {
  try {
    cron.schedule("* * * * *", async () => {
      let { priceFinal, title_def } = await scraperData(
        "https://www.amazon.it/HP-Gaming-braccioli-regolabili-supporto/dp/B07TZD3PFX/ref=sr_1_1?__mk_it_IT=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=1RCHFWA3XUF8K&dchild=1&keywords=poltrona+omen&qid=1627850702&sprefix=poltorna+omen%2Caps%2C193&sr=8-1"
      );
      console.log("data from amazon", priceFinal, title_def);
      priceFinal = parseInt(priceFinal)
      /*  ++count
        if (count === 3) {
          priceFinal = 5320
        }*/
      if (initialPrice === 0) {
        console.log("solo la prima volta");
        initialPrice = parseInt(priceFinal);
        console.log('price from Amazon setted as initial price', initialPrice)
      }

      if (!initialFlow) {
        if (priceFinal !== initialPrice) {
          console.log('prezzo diverso');
          const emailInstance = new EmailManager();
          emailInstance.sendResultFetchDailyData({
            prevPrice: initialPrice,
            priceFinal,
            item: title_def,
          });
          initialPrice = priceFinal;
        }
      }
      initialFlow = false;
      console.log('inizialFlow set at false')
      console.log(`initial price:${initialPrice} - currentAmazonPrice:${priceFinal}`)
    });
  } catch (err) {
    console.log("Error", err);
  }
})();

1 Like

Good morning Enzo, Yesterday I added a Try catch who wrap this

 await puppet.launch({ args: ["--no-sandbox"] });

and I get this log in production:

05 Aug, 08:37:00.881
ASIN=B07TZD3PFX
05 Aug, 08:37:00.881
before puppet.launch
05 Aug, 08:37:00.976
scraperData catch Error: Failed to launch the browser process!
05 Aug, 08:37:00.976
/workspace/node_modules/puppeteer/.local-chromium/linux-884014/chrome-linux/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

05 Aug, 08:37:00.976
05 Aug, 08:37:00.976
05 Aug, 08:37:00.976
TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
05 Aug, 08:37:00.976
05 Aug, 08:37:00.976
at onClose (/workspace/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:194:20)
05 Aug, 08:37:00.976
at Interface.<anonymous> (/workspace/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:184:68)
05 Aug, 08:37:00.976
at Interface.emit (node:events:406:35)
05 Aug, 08:37:00.976
at Interface.close (node:readline:586:8)
05 Aug, 08:37:00.976
at Socket.onend (node:readline:277:10)
05 Aug, 08:37:00.976
at Socket.emit (node:events:406:35)
05 Aug, 08:37:00.976
at endReadableNT (node:internal/streams/readable:1331:12)
05 Aug, 08:37:00.976
at processTicksAndRejections (node:internal/process/task_queues:83:21)

the problems is with puppetter library and chrome:
I have see this https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#chrome-headless-doesnt-launch-on-unix
but no reference is made to how to resolve with Qovery and puppeteer.

1 Like

Hello again !

As i can see you’re using buildpacks but it looks like that you need some specific resources to make pupetter works. Since buildpacks is used for really standard stuff, I think you should try to use a docker file to make it works.

For me it seems to be the best option but I’m not a puppeteer expert.

1 Like

thanks Enzo I’ll try…

Keep my in touch, I’d like to know how you manage it :grinning:

1 Like

Any news on this subject ? pupeeter only have documention for heroku buildpacks