Puppeteer js not work

Questions and Answers Deployment
1

  • OS: ubuntu 20.04

  • Programming language and version: Javascript
    Hi everyone, I have a problem with the puppeteer js library:
    On the local environment everything works, but as soon as I do the deploy on Qovery dotn works.
    Can anybody help me, or give me some advice?
    Thank you!

2

Hello,
can you be a bit more specific ? What’s going wrong ?

1 Like
3

executing this function the script stops at console.log ((“ASIN =” + m [4])
Everything else does not work, but without errors at runtime

const scraperData = async (url) => {
  const regex = RegExp(
    "https://www.amazon.it/([\\w-]+/)?(dp|gp/product)/(\\w+/)?(\\w{10})"
  );
  m = url.match(regex);

  if (m) {
    console.log("ASIN=" + m[4]);
  }

  const browser = await puppet.launch({ args: ["--no-sandbox"] });
  const page = await browser.newPage();
  await page.goto(url);
  await page.waitForXPath('//*[@id="priceblock_ourprice"]');
  const [priceblock_ourprice] = await page.$x('//*[@id="priceblock_ourprice"]');
  const priceTxt = await priceblock_ourprice.getProperty("textContent");
  let priceFinal = await priceTxt.jsonValue();
  priceFinal = priceFinal.replace("€", "").replace(",", "");

  const [title] = await page.$x('//*[@id="productTitle"]');
  let title_def = await title.getProperty("textContent");
  title_def = await title_def.jsonValue();
  title_def = title_def.trim();
  browser.close();
  return { priceFinal, title_def };
};

but it works in the local environment.

1 Like
4

Could you give me the URL of your app ?

1 Like
5

Sorry, where do i find the url in the dashboard?

1 Like
6

It’s the url in your browser when you’re on you app.

1 Like
7

ok:

but is an private page…

8

The url you gave me is linked to a running app and everything looks good. The console.log is well called and return values.

9

ok but the code after the console.log it doesn’t run, that’s the problem.
below all code…

(async () => {
  try {
    cron.schedule("* * * * *", async () => {
      let { priceFinal, title_def } = await scraperData(
        "https://www.amazon.it/HP-Gaming-braccioli-regolabili-supporto/dp/B07TZD3PFX/ref=sr_1_1?__mk_it_IT=%C3%85M%C3%85%C5%BD%C3%95%C3%91&crid=1RCHFWA3XUF8K&dchild=1&keywords=poltrona+omen&qid=1627850702&sprefix=poltorna+omen%2Caps%2C193&sr=8-1"
      );
      console.log("data from amazon", priceFinal, title_def);
      priceFinal = parseInt(priceFinal)
      /*  ++count
        if (count === 3) {
          priceFinal = 5320
        }*/
      if (initialPrice === 0) {
        console.log("solo la prima volta");
        initialPrice = parseInt(priceFinal);
        console.log('price from Amazon setted as initial price', initialPrice)
      }

      if (!initialFlow) {
        if (priceFinal !== initialPrice) {
          console.log('prezzo diverso');
          const emailInstance = new EmailManager();
          emailInstance.sendResultFetchDailyData({
            prevPrice: initialPrice,
            priceFinal,
            item: title_def,
          });
          initialPrice = priceFinal;
        }
      }
      initialFlow = false;
      console.log('inizialFlow set at false')
      console.log(`initial price:${initialPrice} - currentAmazonPrice:${priceFinal}`)
    });
  } catch (err) {
    console.log("Error", err);
  }
})();
1 Like
10

Good morning Enzo, Yesterday I added a Try catch who wrap this

 await puppet.launch({ args: ["--no-sandbox"] });

and I get this log in production:

05 Aug, 08:37:00.881
ASIN=B07TZD3PFX
05 Aug, 08:37:00.881
before puppet.launch
05 Aug, 08:37:00.976
scraperData catch Error: Failed to launch the browser process!
05 Aug, 08:37:00.976
/workspace/node_modules/puppeteer/.local-chromium/linux-884014/chrome-linux/chrome: error while loading shared libraries: libnss3.so: cannot open shared object file: No such file or directory

05 Aug, 08:37:00.976
05 Aug, 08:37:00.976
05 Aug, 08:37:00.976
TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
05 Aug, 08:37:00.976
05 Aug, 08:37:00.976
at onClose (/workspace/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:194:20)
05 Aug, 08:37:00.976
at Interface.<anonymous> (/workspace/node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:184:68)
05 Aug, 08:37:00.976
at Interface.emit (node:events:406:35)
05 Aug, 08:37:00.976
at Interface.close (node:readline:586:8)
05 Aug, 08:37:00.976
at Socket.onend (node:readline:277:10)
05 Aug, 08:37:00.976
at Socket.emit (node:events:406:35)
05 Aug, 08:37:00.976
at endReadableNT (node:internal/streams/readable:1331:12)
05 Aug, 08:37:00.976
at processTicksAndRejections (node:internal/process/task_queues:83:21)

the problems is with puppetter library and chrome:
I have see this https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md#chrome-headless-doesnt-launch-on-unix
but no reference is made to how to resolve with Qovery and puppeteer.

1 Like
11

Hello again !

As i can see you’re using buildpacks but it looks like that you need some specific resources to make pupetter works. Since buildpacks is used for really standard stuff, I think you should try to use a docker file to make it works.

For me it seems to be the best option but I’m not a puppeteer expert.

1 Like
12

thanks Enzo I’ll try…

13

Keep my in touch, I’d like to know how you manage it :grinning:

1 Like
14

Any news on this subject ? pupeeter only have documention for heroku buildpacks