Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
P
Pacemaker Custom Catalog Import CSV Source Transformation
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
Examples
Pacemaker Custom Catalog Import CSV Source Transformation
Commits
6072be84
Commit
6072be84
authored
5 years ago
by
Vadim Justus
Browse files
Options
Downloads
Patches
Plain Diff
Add readme and comments
parent
574e0d08
Branches
master
No related tags found
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
Model/Executor/TransformationExecutor.php
+16
-2
16 additions, 2 deletions
Model/Executor/TransformationExecutor.php
README.md
+60
-0
60 additions, 0 deletions
README.md
with
76 additions
and
2 deletions
Model/Executor/TransformationExecutor.php
+
16
−
2
View file @
6072be84
...
...
@@ -93,35 +93,49 @@ class TransformationExecutor extends AbstractExecutor
foreach
(
$files
as
$filepath
)
{
$fileInfo
=
new
\SplFileInfo
(
$filepath
);
// Ignore the OK file
if
(
strtolower
(
$fileInfo
->
getExtension
())
===
'ok'
)
{
continue
;
}
$fileCount
++
;
// Open a file stream for source CSV file
$sourceFileName
=
$fileInfo
->
getBasename
();
$source
=
$this
->
readFactory
->
create
(
$workingDirectory
.
'/'
.
$sourceFileName
,
DriverPool
::
FILE
);
// Create target file name
$fileCount
++
;
$targetFileName
=
'product-import_'
.
$timestamp
.
'_'
.
str_pad
((
string
)
$fileCount
,
2
,
'0'
,
STR_PAD_LEFT
)
.
'.csv'
;
// Add file name to import file list (this will be written to the OK file as content)
$importFiles
[]
=
$targetFileName
;
$targetPath
=
$workingDirectory
.
'/'
.
$targetFileName
;
// Create an empty CSV file, since writeFactory requires an existing file
$this
->
io
->
write
(
$targetPath
,
''
);
// Open a file stream for target CSV file
$target
=
$this
->
writeFactory
->
create
(
$targetPath
,
DriverPool
::
FILE
,
"w"
);
// Fetch the first row from source file
$header
=
$source
->
readCsv
();
// and write it to the target file, while pass it thru the mapping chain
$target
->
writeCsv
(
$this
->
mapping
->
header
(
$header
));
// Process all the other rows while pass them thru the mapping chain
while
((
$row
=
$source
->
readCsv
())
!==
false
)
{
$target
->
writeCsv
(
$this
->
mapping
->
row
(
array_combine
(
$header
,
$row
)));
}
// close file streams
$source
->
close
();
$target
->
close
();
}
// Create OK file for the import steps, while add a list of import files as content
$okFileName
=
'product-import_'
.
$timestamp
.
'.ok'
;
$this
->
io
->
write
(
$workingDirectory
.
'/'
.
$okFileName
,
implode
(
PHP_EOL
,
$importFiles
));
}
...
...
This diff is collapsed.
Click to expand it.
README.md
0 → 100644
+
60
−
0
View file @
6072be84
# Pacemaker customization example
This example demonstrates
-
how to change a step executor for an existing pipeline
-
how to implement a CSV based data mapping layer for the transformation step
-
how to manipulate default configuration
Please refer also the
[
Pacemaker documentation
](
https://docs.met.tdintern.de/pacemaker/1.2/how-to-extend/import-processes/transform-foreign-import-source.html
)
## The Executor
The executor is the central entry point for this module. The executor will be called by the
[
pipeline runners
](
https://docs.met.tdintern.de/pacemaker/1.2/get-started/how-to-configure-the-runners.html
)
.
To learn how this module works, you should start analysing the code of the executor
`\TechDivision\CsvSourceTransformationExample\Model\Executor\TransformationExecutor`
.
## Mapping Chain
For data mapping this module introduces a chain. Therefore an interface
`\TechDivision\CsvSourceTransformationExample\Api\MappingInterface`
is defined. There is a default
preference given in the
`di.xml`
of this module:
```
xml
<preference
for=
"TechDivision\CsvSourceTransformationExample\Api\MappingInterface"
type=
"TechDivision\CsvSourceTransformationExample\Model\MappingChain"
/>
```
If you look into the
`\TechDivision\CsvSourceTransformationExample\Model\MappingChain`
you will see, that this
class delegates the whole logic to set of mapper instances, which are injected via DI. This approach make the chain
very flexible and extendable, since you just need to add additional mapper via DI.
```
xml
<type
name=
"TechDivision\CsvSourceTransformationExample\Model\MappingChain"
>
<arguments>
<argument
name=
"mapper"
xsi:type=
"array"
>
<item
name=
"categories"
xsi:type=
"object"
>
TechDivision\CsvSourceTransformationExample\Model\Mapping\Categories
</item>
<item
name=
"website_code"
xsi:type=
"object"
>
TechDivision\CsvSourceTransformationExample\Model\Mapping\WebsiteCode
</item>
</argument>
</arguments>
</type>
```
This example module defines two mapper. Each mapper is responsible for one dedicated value type e.g. categories or
website code. This is also an advantage of the used chain pattern.
## The Transformation
You'll find some example CSV files in the
`test-files`
directory of this module. We need to change the column key
for
`categories`
, which is
`cat_path`
in the given file. And there is a value
`_DEFAULT_`
in the column
`product_websites`
, which needs to be replaces by the default website code. The provided category path is also
incomplete since there is not
`Default Category`
as the category root.
## The Configuration
This module also changes the default configuration for the catalog import by manipulating the file pattern in
`etc/config.xml`
.
```
xml
<file_name_pattern>
<![CDATA[/pim-source_(?P<identifier>[0-9a-z\-]*)([_0-9]*?).(csv|ok)/i]]>
</file_name_pattern>
```
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment