Skip to content

Commit 33f074e

Browse files
sgiehldiosmosis
sgiehl
authored and
diosmosis
committed
Keep flattened columns as extra columns (matomo-org#12199)
* Add reports dimensions to metadata of report and rows * translate dimension columns * updates test files * fix possible error when no report is available * update tests * Improve subdimension detection * Adjust tests for labelX logic * Makes flattener compatible with 3 dimensions * Adds new method getThirdLeveltableDimension to report class * Do not ask for 2fa authentication code when CoreUpdater is being requested (matomo-org#13796) Could fix an edge case where user is logged in, but hasn't confirmed the auth code (so the user is not actually logged in), and then an update appears. * Added Fallback Method for Alexa in SEO Plugin (matomo-org#13552) * added fallback method for Alexa, fixes issue matomo-org#13427 * do not use short array syntax for consistency with other methods * use mini link for Alexa, use DomXPath to filter out the global ranking instead of regex * Use db sessions by default, deprecate file session handler (matomo-org#13540) * use db sessions by default, deprecate file session handler * trying to fix tests * Prevent trigger errors on demand for instances that are opened to anonymous (matomo-org#13535) fix matomo-org#13513 * Remove the previous exception in base validator so the same error is not printed twice (matomo-org#13801) * Fixing build (matomo-org#13802) * update submodule * Update screenshots and try to get test to pass. * Get SingleMetricView to pass. (matomo-org#13803) * Quickform2 throws warnings with PHP7.2 (matomo-org#13463) fixes matomo-org#13272 Haven't actually tested it but should fix the issue. If tests pass, the logic would be still the same. I don't have a PHP 7.2 running here otherwise at the moment * Send bulk requests in chunks when needed (matomo-org#13444) * send bulk requests in chunks * send requests correctly * Make log and report data screen less technical (matomo-org#13464) * When you are logged out, the URL gets lost when you log in (matomo-org#13441) It won't remember any hash as the hash won't be visible in the referrer etc but it would work for most other pages. To make it work for hash it would get likely way more complicated like we would need to persist it through JS, temporarily store it somewhere and redirect accordingly. It fixes the case mentioned in the issue. fix matomo-org#13328 * show full information of URL only on extra click (matomo-org#13585) * Add option to opt in to use send beacon (matomo-org#13451) * Add option to opt in to use send beacon * Fix JS tracker test. * do not overrwite existing subrow metadata * update test files
1 parent df062ae commit 33f074e

47 files changed

Lines changed: 997 additions & 17 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎core/API/DataTableManipulator.php‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,8 +33,7 @@ abstract class DataTableManipulator
3333
protected $apiModule;
3434
protected $apiMethod;
3535
protected $request;
36-
37-
private $apiMethodForSubtable;
36+
protected $apiMethodForSubtable;
3837

3938
/**
4039
* Constructor
@@ -144,7 +143,7 @@ abstract protected function manipulateSubtableRequest($request);
144143
* @throws Exception
145144
* @return string
146145
*/
147-
private function getApiMethodForSubtable($request)
146+
protected function getApiMethodForSubtable($request)
148147
{
149148
if (!$this->apiMethodForSubtable) {
150149
if (!empty($request['idSite'])) {

‎core/API/DataTableManipulator/Flattener.php‎

Lines changed: 55 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@
1212
use Piwik\Common;
1313
use Piwik\DataTable;
1414
use Piwik\DataTable\Row;
15+
use Piwik\Plugin\ReportsProvider;
1516

1617
/**
1718
* This class is responsible for flattening data tables.
@@ -70,32 +71,43 @@ protected function manipulateDataTable($dataTable)
7071
$dataTable->filter('ReplaceSummaryRowLabel');
7172
$dataTable->filter('ReplaceColumnNames');
7273

73-
$this->flattenDataTableInto($dataTable, $newDataTable);
74+
$report = ReportsProvider::factory($this->apiModule, $this->apiMethod);
75+
if (!empty($report)) {
76+
$dimension = $report->getDimension();
77+
}
78+
79+
$dimensionName = !empty($dimension) ? str_replace('.', '_', $dimension->getId()) : 'label1';
80+
81+
$this->flattenDataTableInto($dataTable, $newDataTable, $level = 1, $dimensionName);
7482

7583
return $newDataTable;
7684
}
7785

7886
/**
7987
* @param $dataTable DataTable
8088
* @param $newDataTable
89+
* @param $dimensionName
8190
*/
82-
protected function flattenDataTableInto($dataTable, $newDataTable, $prefix = '', $logo = false)
91+
protected function flattenDataTableInto($dataTable, $newDataTable, $level, $dimensionName, $prefix = '', $logo = false)
8392
{
8493
foreach ($dataTable->getRows() as $rowId => $row) {
85-
$this->flattenRow($row, $rowId, $newDataTable, $prefix, $logo);
94+
$this->flattenRow($row, $rowId, $newDataTable, $level, $dimensionName, $prefix, $logo);
8695
}
8796
}
8897

8998
/**
9099
* @param Row $row
91100
* @param DataTable $dataTable
92101
* @param string $labelPrefix
102+
* @param string $dimensionName
93103
* @param bool $parentLogo
94104
*/
95-
private function flattenRow(Row $row, $rowId, DataTable $dataTable,
105+
private function flattenRow
106+
(Row $row, $rowId, DataTable $dataTable, $level, $dimensionName,
96107
$labelPrefix = '', $parentLogo = false)
97108
{
98-
$label = $row->getColumn('label');
109+
$origLabel = $label = $row->getColumn('label');
110+
99111
if ($label !== false) {
100112
$label = trim($label);
101113

@@ -107,8 +119,16 @@ private function flattenRow(Row $row, $rowId, DataTable $dataTable,
107119
}
108120
}
109121

122+
$origLabel = $label;
123+
110124
$label = $labelPrefix . $label;
111125
$row->setColumn('label', $label);
126+
127+
if ($row->getMetadata($dimensionName)) {
128+
$origLabel = $row->getMetadata($dimensionName) . $this->recursiveLabelSeparator . $origLabel;
129+
}
130+
131+
$row->setMetadata($dimensionName, $origLabel);
112132
}
113133

114134
$logo = $row->getMetadata('logo');
@@ -140,7 +160,36 @@ private function flattenRow(Row $row, $rowId, DataTable $dataTable,
140160
$dataTable->addRow($row);
141161
}
142162
$prefix = $label . $this->recursiveLabelSeparator;
143-
$this->flattenDataTableInto($subTable, $dataTable, $prefix, $logo);
163+
164+
$report = ReportsProvider::factory($this->apiModule, $this->apiMethod);
165+
if (!empty($report)) {
166+
$subDimension = $report->getSubtableDimension();
167+
}
168+
169+
if ($level === 2) {
170+
$subDimension = $report->getThirdLeveltableDimension();
171+
}
172+
173+
if (empty($subDimension)) {
174+
$report = ReportsProvider::factory($this->apiModule, $this->getApiMethodForSubtable($this->request));
175+
$subDimension = $report->getDimension();
176+
}
177+
178+
$subDimensionName = $subDimension ? str_replace('.', '_', $subDimension->getId()) : 'label' . (substr_count($prefix, $this->recursiveLabelSeparator) + 1);
179+
180+
if ($origLabel !== false) {
181+
foreach ($subTable->getRows() as $subRow) {
182+
foreach ($row->getMetadata() as $name => $value) {
183+
if ($subRow->getMetadata($name) === false) {
184+
$subRow->setMetadata($name, $value);
185+
}
186+
}
187+
188+
$subRow->setMetadata($dimensionName, $origLabel);
189+
}
190+
}
191+
192+
$this->flattenDataTableInto($subTable, $dataTable, $level + 1, $subDimensionName, $prefix, $logo);
144193
}
145194
}
146195

‎core/DataTable/Renderer.php‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@
99
namespace Piwik\DataTable;
1010

1111
use Exception;
12+
use Piwik\Columns\Dimension;
1213
use Piwik\Common;
1314
use Piwik\DataTable;
1415
use Piwik\Metrics;
@@ -230,6 +231,15 @@ protected function translateColumnNames($names)
230231
}
231232
}
232233

234+
foreach (Dimension::getAllDimensions() as $dimension) {
235+
$dimensionId = str_replace('.', '_', $dimension->getId());
236+
$dimensionName = $dimension->getName();
237+
238+
if (!empty($dimensionId) && !empty($dimensionName)) {
239+
$t[$dimensionId] = $dimensionName;
240+
}
241+
}
242+
233243
$this->columnTranslations = & $t;
234244
}
235245

‎core/Metrics.php‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,6 @@
99
namespace Piwik;
1010

1111
use Piwik\Cache as PiwikCache;
12-
use Piwik\Metrics\Formatter;
1312

1413
require_once PIWIK_INCLUDE_PATH . "/core/Piwik.php";
1514

‎core/Plugin/Report.php‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -594,6 +594,12 @@ protected function buildReportMetadata()
594594
$report['isSubtableReport'] = $this->isSubtableReport;
595595
}
596596

597+
$dimensions = $this->getDimensions();
598+
599+
if (count($dimensions) > 1) {
600+
$report['dimensions'] = $dimensions;
601+
}
602+
597603
$report['metrics'] = $this->getMetrics();
598604
$report['metricsDocumentation'] = $this->getMetricsDocumentation();
599605
$report['processedMetrics'] = $this->getProcessedMetrics();
@@ -745,6 +751,34 @@ public function getDimension()
745751
return $this->dimension;
746752
}
747753

754+
/**
755+
* Get dimensions used for current report and its subreports
756+
*
757+
* @return array [dimensionId => dimensionName]
758+
* @ignore
759+
*/
760+
public function getDimensions()
761+
{
762+
$dimensions = [];
763+
764+
if (!empty($this->getDimension())) {
765+
$dimensionId = str_replace('.', '_', $this->getDimension()->getId());
766+
$dimensions[$dimensionId] = $this->getDimension()->getName();
767+
}
768+
769+
if (!empty($this->getSubtableDimension())) {
770+
$subDimensionId = str_replace('.', '_', $this->getSubtableDimension()->getId());
771+
$dimensions[$subDimensionId] = $this->getSubtableDimension()->getName();
772+
}
773+
774+
if (!empty($this->getThirdLeveltableDimension())) {
775+
$subDimensionId = str_replace('.', '_', $this->getThirdLeveltableDimension()->getId());
776+
$dimensions[$subDimensionId] = $this->getThirdLeveltableDimension()->getName();
777+
}
778+
779+
return $dimensions;
780+
}
781+
748782
/**
749783
* Returns the order of the report
750784
* @return int
@@ -788,6 +822,36 @@ public function getSubtableDimension()
788822
return $subtableReport->getDimension();
789823
}
790824

825+
/**
826+
* Returns the Dimension instance of the subtable report of this report's subtable report.
827+
*
828+
* @return Dimension|null The subtable report's dimension or null if there is no subtable report or
829+
* no dimension for the subtable report.
830+
* @api
831+
*/
832+
public function getThirdLeveltableDimension()
833+
{
834+
if (empty($this->actionToLoadSubTables)) {
835+
return null;
836+
}
837+
838+
list($subtableReportModule, $subtableReportAction) = $this->getSubtableApiMethod();
839+
840+
$subtableReport = ReportsProvider::factory($subtableReportModule, $subtableReportAction);
841+
if (empty($subtableReport) || empty($subtableReport->actionToLoadSubTables)) {
842+
return null;
843+
}
844+
845+
list($subSubtableReportModule, $subSubtableReportAction) = $subtableReport->getSubtableApiMethod();
846+
847+
$subSubtableReport = ReportsProvider::factory($subSubtableReportModule, $subSubtableReportAction);
848+
if (empty($subSubtableReport)) {
849+
return null;
850+
}
851+
852+
return $subSubtableReport->getDimension();
853+
}
854+
791855
/**
792856
* Returns true if the report is for another report's subtable, false if otherwise.
793857
*

‎plugins/API/ProcessedReport.php‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -428,6 +428,7 @@ private function handleTableReport($idSite, $dataTable, &$reportMetadata, $showR
428428
$columns = @$reportMetadata['metrics'] ?: array();
429429

430430
if ($hasDimension) {
431+
431432
$columns = array_merge(
432433
array('label' => $reportMetadata['dimension']),
433434
$columns

‎tests/PHPUnit/System/FlattenReportsTest.php‎

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,9 @@
77
*/
88
namespace Piwik\Tests\System;
99

10+
use Piwik\EventDispatcher;
11+
use Piwik\Piwik;
12+
use Piwik\Plugins\Referrers\Reports\GetWebsites;
1013
use Piwik\Tests\Framework\TestCase\SystemTestCase;
1114
use Piwik\Tests\Fixtures\ManyVisitsWithSubDirReferrersAndCustomVars;
1215

@@ -25,6 +28,17 @@ class FlattenReportsTest extends SystemTestCase
2528
*/
2629
public function testApi($api, $params)
2730
{
31+
EventDispatcher::getInstance()->addObserver('Report.filterReports', function(&$reports) {
32+
$newReports = [];
33+
foreach ($reports as $report) {
34+
if ($report instanceof GetWebsites) {
35+
continue;
36+
}
37+
$newReports[] = $report;
38+
}
39+
$newReports[] = new DimensionLessReport();
40+
$reports = $newReports;
41+
});
2842
$this->runApiTests($api, $params);
2943
}
3044

@@ -125,4 +139,16 @@ public static function getOutputPrefix()
125139
}
126140
}
127141

128-
FlattenReportsTest::$fixture = new ManyVisitsWithSubDirReferrersAndCustomVars();
142+
FlattenReportsTest::$fixture = new ManyVisitsWithSubDirReferrersAndCustomVars();
143+
144+
145+
class DimensionLessReport extends GetWebsites
146+
{
147+
protected function init()
148+
{
149+
parent::init();
150+
$this->dimension = null;
151+
$this->module = 'Referrers';
152+
$this->action = 'GetWebsites';
153+
}
154+
}

‎tests/PHPUnit/System/expected/test_BackwardsCompatibility1XTest_flat__Actions.getPageUrls_range.xml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@
1515
<exit_rate>0%</exit_rate>
1616
<avg_time_generation>0.224</avg_time_generation>
1717
<url>http://example.org/</url>
18+
<Actions_PageUrl>/</Actions_PageUrl>
1819
<segment>pageUrl==http%3A%2F%2Fexample.org%2F</segment>
1920
</row>
2021
<row>
@@ -34,6 +35,7 @@
3435
<exit_rate>100%</exit_rate>
3536
<avg_time_generation>0.023</avg_time_generation>
3637
<url>http://example.org/index.htm</url>
38+
<Actions_PageUrl>/index.htm</Actions_PageUrl>
3739
<segment>pageUrl==http%3A%2F%2Fexample.org%2Findex.htm</segment>
3840
</row>
3941
<row>
@@ -56,6 +58,7 @@
5658
<exit_rate>0%</exit_rate>
5759
<avg_time_generation>0.234</avg_time_generation>
5860
<url>http://example.org/index.htm?parameter=Should display</url>
61+
<Actions_PageUrl>/index.htm?parameter=Should display</Actions_PageUrl>
5962
<segment>pageUrl==http%3A%2F%2Fexample.org%2Findex.htm%3Fparameter%3DShould+display</segment>
6063
</row>
6164
<row>
@@ -80,6 +83,7 @@
8083
<exit_rate>100%</exit_rate>
8184
<avg_time_generation>0.134</avg_time_generation>
8285
<url>http://example.org/store/purchase.htm</url>
86+
<Actions_PageUrl>/store/purchase.htm</Actions_PageUrl>
8387
<segment>pageUrl==http%3A%2F%2Fexample.org%2Fstore%2Fpurchase.htm</segment>
8488
</row>
8589
</result>

0 commit comments

Comments
 (0)