Computing code coverage on Azure Pipelines

  107113
September 15, 2019 17:28 nikita.ppv@gmail.com (Nikita Popov)
Hi,

gcov.php.net used to provide valgrind reports and code coverage for
php-src. It no longer works with 7.4/8.0, because the OS is very old, and
building new versions of PHP there gets very hard. The valgrind reports
have essentially been subsumed by asan/ubsan on Azure Pipelines, but we're
still missing a replacement for code coverage.

I think it should be possible to set up an Azure Pipelines job that runs
tests with coverage and publishes it, preferably on Azure Pipelines itself,
or if that doesn't work something like codecov.io.

Maybe someone is interested in giving this a try?

Regards,
Nikita
  107114
September 15, 2019 17:38 carusogabriel34@gmail.com (Gabriel Caruso)
Hello Nikita,

Em dom, 15 de set de 2019 às 19:29, Nikita Popov ppv@gmail.com>
escreveu:

> Hi, > > gcov.php.net used to provide valgrind reports and code coverage for > php-src. It no longer works with 7.4/8.0, because the OS is very old, and > building new versions of PHP there gets very hard. The valgrind reports > have essentially been subsumed by asan/ubsan on Azure Pipelines, but we're > still missing a replacement for code coverage. > > I think it should be possible to set up an Azure Pipelines job that runs > tests with coverage and publishes it, preferably on Azure Pipelines itself, > or if that doesn't work something like codecov.io. > > Maybe someone is interested in giving this a try? > > Regards, > Nikita >
I'm, as I've also reported this a while ago: https://bugs.php.net/bug.php?id=78288. Do you have something in mind how/what to set up? Best regards, -- Gabriel Caruso
  107115
September 15, 2019 18:35 nikita.ppv@gmail.com (Nikita Popov)
On Sun, Sep 15, 2019 at 7:39 PM Gabriel Caruso <carusogabriel34@gmail.com>
wrote:

> Hello Nikita, > > Em dom, 15 de set de 2019 às 19:29, Nikita Popov ppv@gmail.com> > escreveu: > >> Hi, >> >> gcov.php.net used to provide valgrind reports and code coverage for >> php-src. It no longer works with 7.4/8.0, because the OS is very old, and >> building new versions of PHP there gets very hard. The valgrind reports >> have essentially been subsumed by asan/ubsan on Azure Pipelines, but we're >> still missing a replacement for code coverage. >> >> I think it should be possible to set up an Azure Pipelines job that runs >> tests with coverage and publishes it, preferably on Azure Pipelines >> itself, >> or if that doesn't work something like codecov.io. >> >> Maybe someone is interested in giving this a try? >> >> Regards, >> Nikita >> > > I'm, as I've also reported this a while ago: > https://bugs.php.net/bug.php?id=78288. > > Do you have something in mind how/what to set up? >
The first step is probably to get code coverage working locally. We have a ../configure --enable-gcov option and there's some extra make targets in build/Makefile.gcov. Maybe taking a look at what https://github.com/php/web-gcov/blob/master/cron/cron.sh does would help. Second step is to add a new job in azure by copying https://github.com/php/php-src/blob/master/azure/job.yml and adding the necessary gcov/lcov magic there. The hard part if how to get those results published on Azure under the "Code coverage" tab. There is a PublishCodeCoverageResults task for this purpose, but I don't know how one would use that with lcov in particular. It seems to me that part of it requires a coverage overview in a specific format and part is just a directory with HTML, so maybe the result from ltp genhtml can just be used there? Figuring out how to make this work is likely going to be "fun" ;) Nikita
>