promjet-stats

Statistics of visitors from some site compaired to Yandex.Metrika or Google Analitycs data.

Vote for us at The Contest!
This is our first apllication.
We will be very thankfull for your support and feedback.

This application arose from the need to control the number of visitors to the client’s site (promjet.ru).
External visitor tracking systems (Yandex.Metrica, Google Analytics) show slightly distorted indicators, because the Firefox browser and some of its branches block tracking tools, which include metrics systems, as well as ad blockers block these services.

We decided to rely on data from the Apache web server log files, they also record all information about visitors.
In addition, we can see the load created by search robots and content indexing systems.
image (1)

Links to actual demos attached in Examples .

Requirements

Access to hosting via FTP or SSH
Access to logfiles from php scripts
Oauth token for Yandex.Metrika access

Data source

First, you need to place a specially written PHP script on the hosting that takes data from log files and converts them to CSV format.
This file must be uploaded to webserver in folder “/api/get_scv/” from root of the site.
As an authentic tool we use token, sent via GET request (added in URL).
This token and filename for logfile is stored in the beginnig of index.php file.

On IRIS side this token was stored in “iris.script” file previosly.

Later we replaced it by “csvgen” package and by command for importing data from website:

zpm "install csvgen"
d ##class(community.csvgen).GenerateFromURL("https://promjet.ru/api/get_csv/?token=6ab225b4-43b5-410a-ac9a-11c1dc2a8c6c",",","dc.irisbi.Promjet")

Data from Yandex Metrika also needs special PHP file stored on hosting.
Copy source file from “/api/get_csv_YaMetrika/” to your hosting root folder and change values in KEY and OAuthToken variables fitted to your projects.

Cube

This data is taken by IRIS, builds a cube and outputs it to DeepSee or DSW.
The second Cube is built in AtScale

Examples

Currently IRIS BI available at this demo site

Data from IRIS goes to AtScale server deployed at ISC AtScale Server.
There builds a cube, merging data from hosting and from Yandex.Metrika

Dashboard on PowerBI get data from the cube and display it in widgets.
Application published under this link
Also we have an interactive page on our site with built-in PowerBI dashboards there

Filters works by click on value

Tableau version accessible with
This Public Tableau Page 1
This Public Tableau Page 2

Logi version of reports is also in repo.
Screenshots attached below:
image (35)
image (37)
Logi server deployed at http://atscale.teccod.com:8888/jinfonet/welcome.jsp