Скачать или смотреть Iceberg Community Sync - Jun 28 2023

Highlights
OOM fix caused by Avro decoder caching (https://github.com/apache/iceberg/pul...)
Multiple shuffle partitions per file (https://github.com/apache/iceberg/pul...)
Python: Add positional deletes (https://github.com/apache/iceberg/pul...)
Python: alter table in transactions (https://github.com/apache/iceberg/pul...)
View metadata implementation (https://github.com/apache/iceberg/pul...)
View support for InMemoryCatalog (https://github.com/apache/iceberg/pul...)
API for multi-table commits (https://github.com/apache/iceberg/pul...)
Catalog Transaction API is still in draft (https://github.com/apache/iceberg/pul...)
Flink: Split ordering based on Sequence Number (https://github.com/apache/iceberg/pul...)
Adding new Iceberg Events Calendar including this sync. (https://calendar.google.com/calendar/...)
Any PMC that wants access to edit this calendar reach out to me on Slack/Dev list.

Releases
Python 0.4 vote is out! (Please verify: https://lists.apache.org/thread/ccj4k...)

Discussion
Bloom/cuckoo/other filters in manifest
Priority-based commit
Documentation efforts #documentation (https://slack.com/app_redirect?channe...)
Proposed updates to the docs site (https://docs.google.com/document/d/1Y...)


AI-generated chapter summaries:
0:00 Chapter 1
Daniel, Brian, and the team discussed various updates and progress made in the past few weeks, including fixes for memory issues, improvements in shuffle partitions for file compaction, positional delete support in Python, and the implementation of view metadata. They also mentioned plans for engine integrations and the need for further discussions on multi-table transactions.

10:29 Chapter 2
The team discussed the progress and next steps for implementing multi-table commits and improving support for ordered read from iceberg tables. They also discussed plans to make the Iceberg community meetings more accessible by providing a public link instead of requiring a subscription to the dev list.

21:03 Chapter 3
The team discussed the new features and improvements included in the ODOP release, such as positional leads, SQL style filters, and performance enhancements. They also explored the possibility of adding Bloom filters to the manifest file to improve point query performance

31:54 Chapter 4
The discussion revolved around the potential use of Parquet Bloom filters to improve performance and reduce costs. They also discussed the possibility of introducing priority-based commit and the challenges associated with it.

42:00 Chapter 5
The team discussed the possibility of implementing a rollback feature in the catalog and snapshot producer. They also talked about prioritizing documentation updates and planning to add examples and tutorials in the future.