Скачать или смотреть How to Filter Empty SKUs and Remove Duplicates in DataWeave

Learn how to effectively filter out entries with empty SKUs and remove duplicates based on order numbers in DataWeave.
---
This video is based on the question https://stackoverflow.com/q/72037338/ asked by the user 'Jesus Abraham Felix González' ( https://stackoverflow.com/u/18901832/ ) and on the answer https://stackoverflow.com/a/72038296/ provided by the user 'Salim Khan' ( https://stackoverflow.com/u/3081578/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: How to filter the sku empty and remove all the coincidense that the same order number

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
How to Filter Empty SKUs and Remove Duplicates in DataWeave

When working with arrays of data, especially in contexts like MuleSoft and DataWeave, you often face situations where you need to clean up your data. One common problem is needing to filter out any records with empty SKUs and also remove duplicates based on their order numbers. In this guide, we’ll explore a solution to this particular challenge step-by-step.

The Problem

You will often receive data that looks like this:

[[See Video to Reveal this Text or Code Snippet]]

From this JSON array, we want to accomplish two things:

Filter out any items with empty SKUs.

Remove duplicate order numbers, ensuring each order number appears only once, provided there is a valid SKU associated with it.

The expected output from the above input looks like this:

[[See Video to Reveal this Text or Code Snippet]]

The Solution

Step 1: Identify Empty SKUs

The first step in cleaning up this data is to identify which order numbers are associated with empty SKUs. We can achieve this with a filter function that checks for empty SKUs and collects their respective order numbers.

Step 2: Filter the Data

After we have a list of order numbers with empty SKUs, the next step is to filter through the original array and exclude any records that match those empty SKU order numbers.

Implementation in DataWeave

Here’s a simple script that accomplishes the above steps using DataWeave:

[[See Video to Reveal this Text or Code Snippet]]

Explanation of the Code

Filtering for Empty SKUs:

payload filter ($.sku == ""): This line iterates over the payload and selects only the elements with an empty sku.

Extracting Order Numbers:

..number: This notation extracts the number field from the filtered results, resulting in an array of order numbers that have empty SKUs.

Final Filter:

payload filter (!(atleastOneEmptySku contains $.number)): This line filters the original payload, retaining only those entries whose number is not in the list of empty SKU numbers.

Conclusion

By breaking the problem into two manageable parts and using the right DataWeave functions, you can effectively clean and filter your data to meet your project requirements. This method not only enhances the quality of your datasets but also ensures that your data processing flows remain efficient and robust.

If you found this guide helpful, consider exploring further on how DataWeave can simplify your data manipulation tasks in MuleSoft!