Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/.licenserc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ header:
- 'LICENSE'
- 'NOTICE'
- 'requirements.txt'
- 'src/iceberg/expected.h'
- 'src/iceberg/util/murmurhash3_internal.*'
- 'src/iceberg/test/resources/**'
- 'src/iceberg/catalog/hive/gen-cpp/**'
Expand Down
25 changes: 22 additions & 3 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,12 +78,19 @@ jobs:
with:
key-prefix: sccache-test-ubuntu-${{ matrix.cmake_build_type }}
job-status: ${{ job.status }}
- name: Build Example
- name: Build Example (C++20)
shell: bash
env:
CC: gcc-14
CXX: g++-14
run: ci/scripts/build_example.sh $(pwd)/example
- name: Build Example (C++23)
shell: bash
env:
CC: gcc-14
CXX: g++-14
ICEBERG_EXAMPLE_CXX_STANDARD: 23
run: ci/scripts/build_example.sh $(pwd)/example
hive:
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.draft == false }}
name: AMD64 Ubuntu 26.04 Hive
Expand Down Expand Up @@ -153,9 +160,14 @@ jobs:
with:
key-prefix: sccache-test-macos
job-status: ${{ job.status }}
- name: Build Example
- name: Build Example (C++20)
shell: bash
run: ci/scripts/build_example.sh $(pwd)/example
- name: Build Example (C++23)
shell: bash
env:
ICEBERG_EXAMPLE_CXX_STANDARD: 23
run: ci/scripts/build_example.sh $(pwd)/example
windows:
if: ${{ github.event_name != 'pull_request' || github.event.pull_request.draft == false }}
name: AMD64 Windows 2025
Expand Down Expand Up @@ -199,8 +211,15 @@ jobs:
with:
key-prefix: sccache-test-windows
job-status: ${{ job.status }}
- name: Build Example
- name: Build Example (C++20)
shell: pwsh
run: |
$ErrorActionPreference = "Stop"
bash -lc 'ci/scripts/build_example.sh $(pwd)/example'
- name: Build Example (C++23)
shell: pwsh
env:
ICEBERG_EXAMPLE_CXX_STANDARD: 23
run: |
$ErrorActionPreference = "Stop"
bash -lc 'ci/scripts/build_example.sh $(pwd)/example'
28 changes: 28 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,34 @@ License: https://www.apache.org/licenses/LICENSE-2.0

--------------------------------------------------------------------------------

This product includes code from zeus-cpp/expected.

* src/iceberg/expected.h is adapted from zeus-cpp/expected.

Copyright: 2024 zeus-cpp.
Home page: https://github.com/zeus-cpp/expected
License: MIT

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

--------------------------------------------------------------------------------

This product bundles utf8proc, which is available under the MIT License:

utf8proc is a software package originally developed by Jan Behrens and the rest
Expand Down
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,14 @@ C++ implementation of [Apache Iceberg™](https://iceberg.apache.org/).

**Required:**

- C++23 compliant compiler (GCC 14+, Clang 18+, MSVC 2022+)
- C++23 compliant compiler (GCC 14+, Clang 18+, MSVC 2022+) to build iceberg-cpp itself
- CMake 3.25+
- [Ninja](https://ninja-build.org/) (recommended build backend)

**Using iceberg-cpp from your project:** the installed public headers require
C++20 at minimum, so applications that link against iceberg-cpp can be
compiled as C++20 or later. The library itself is still built as C++23.

**Optional:**

- Python 3 and [pre-commit](https://pre-commit.com/) (for linting)
Expand Down Expand Up @@ -121,6 +125,14 @@ If you are using provided Apache Arrow, include `/path/to/arrow` in `CMAKE_PREFI
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH="/path/to/install;/path/to/arrow"
```

The examples build as C++20 by default, which is the minimum standard supported
by the public headers. Set `ICEBERG_EXAMPLE_CXX_STANDARD` to `23` to build them
as C++23 instead:

```bash
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH=/path/to/install -DICEBERG_EXAMPLE_CXX_STANDARD=23
```

## Customizing Dependency URLs

If you experience network issues when downloading dependencies, you can customize the download URLs using environment variables:
Expand Down
2 changes: 2 additions & 0 deletions ci/scripts/build_example.sh
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ set -eux
source_dir=${1}
build_dir=${1}/build
run_example=${ICEBERG_RUN_EXAMPLE:-OFF}
cxx_standard=${ICEBERG_EXAMPLE_CXX_STANDARD:-20}

# Clean up before configuring. If Windows still holds a just-built exe/dll
# after the retries, let mkdir fail rather than reuse a half-deleted tree.
Expand Down Expand Up @@ -53,6 +54,7 @@ fi

build_type="${ICEBERG_BUILD_TYPE:-Debug}"
CMAKE_ARGS+=("-DCMAKE_BUILD_TYPE=${build_type}")
CMAKE_ARGS+=("-DICEBERG_EXAMPLE_CXX_STANDARD=${cxx_standard}")

cmake "${CMAKE_ARGS[@]}" ${source_dir}
cmake --build .
Expand Down
76 changes: 71 additions & 5 deletions example/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,79 @@ cmake_minimum_required(VERSION 3.25)

project(example)

set(CMAKE_CXX_STANDARD 23)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make C++23 still as default and let it accept user supplied option so that C++20 can be test manually locally.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhjwpku Thank you for the comments.

Making it configurable is a good idea. Have different thoughts on making C++23 as default though.

With this patch, it changes the minimum supported C++ standard from C++23 to C++20 for downstream consumers. And C++20 is the interface contract between the consumer and iceberg-cpp library, and the contract should be tested continuously.

Setting the default to C++20 ensures the minimum supported standard (contract) is continuously exercised. Defaulting it to C++23 would let C++20 only breakages slip through.

One refinement is that C++23 compatibility should still be tested separately. C++23 should accepts C++20 code, but we can enhance this by provide an optional example configuration for C++23, for example, expose an ICEBERG_EXAMPLE_CXX_STANDARD cache setting that defaults to 20 and accepts 23; then update CI to build both.
And also refine the document to state clearly that the minimum C++ standard is C++20 for public headers. What do you think?

Happy to make changes either way.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sense to me, I think we should build both for compatibility purpose.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhjwpku Exposed ICEBERG_EXAMPLE_CXX_STANDARD and updated document accordingly in de7b278.

# C++20 is the minimum standard iceberg-cpp's public headers support, so the
# example builds as C++20 by default to keep that contract exercised. Set this to
# 23 to also check the headers from a C++23 consumer.
set(ICEBERG_EXAMPLE_CXX_STANDARD
20
CACHE STRING "C++ standard used to build the example (20 or 23)")
set_property(CACHE ICEBERG_EXAMPLE_CXX_STANDARD PROPERTY STRINGS 20 23)
if(NOT ICEBERG_EXAMPLE_CXX_STANDARD MATCHES "^(20|23)$")
message(FATAL_ERROR "ICEBERG_EXAMPLE_CXX_STANDARD must be 20 or 23, got "
"'${ICEBERG_EXAMPLE_CXX_STANDARD}'")
endif()

set(CMAKE_CXX_STANDARD ${ICEBERG_EXAMPLE_CXX_STANDARD})
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

find_package(iceberg CONFIG REQUIRED COMPONENTS bundle rest)

if(TARGET iceberg::iceberg_bundle_shared)
set(ICEBERG_BUNDLE_TARGET iceberg::iceberg_bundle_shared)
else()
set(ICEBERG_BUNDLE_TARGET iceberg::iceberg_bundle_static)
endif()

if(TARGET iceberg::iceberg_rest_shared)
set(ICEBERG_REST_TARGET iceberg::iceberg_rest_shared)
else()
set(ICEBERG_REST_TARGET iceberg::iceberg_rest_static)
endif()

add_executable(demo_example demo_example.cc)

target_link_libraries(demo_example
PRIVATE "$<IF:$<TARGET_EXISTS:iceberg::iceberg_bundle_shared>,iceberg::iceberg_bundle_shared,iceberg::iceberg_bundle_static>"
"$<IF:$<TARGET_EXISTS:iceberg::iceberg_rest_shared>,iceberg::iceberg_rest_shared,iceberg::iceberg_rest_static>"
)
target_link_libraries(demo_example PRIVATE ${ICEBERG_BUNDLE_TARGET}
${ICEBERG_REST_TARGET})

# Compile every installed public header as a consumer using
# ICEBERG_EXAMPLE_CXX_STANDARD. The installed include
# tree is the public API contract: iceberg_install_all_headers excludes internal
# headers before packaging them.
get_target_property(ICEBERG_BUNDLE_INCLUDE_DIRS ${ICEBERG_BUNDLE_TARGET}
INTERFACE_INCLUDE_DIRECTORIES)
foreach(ICEBERG_INCLUDE_DIR IN LISTS ICEBERG_BUNDLE_INCLUDE_DIRS)
if(EXISTS "${ICEBERG_INCLUDE_DIR}/iceberg")
set(ICEBERG_PUBLIC_INCLUDE_DIR "${ICEBERG_INCLUDE_DIR}")
break()
endif()
endforeach()

if(NOT ICEBERG_PUBLIC_INCLUDE_DIR)
message(FATAL_ERROR "Could not locate iceberg's installed public headers")
endif()

file(GLOB_RECURSE
ICEBERG_PUBLIC_HEADERS
CONFIGURE_DEPENDS
"${ICEBERG_PUBLIC_INCLUDE_DIR}/iceberg/*.h"
"${ICEBERG_PUBLIC_INCLUDE_DIR}/iceberg/*.hpp")
list(SORT ICEBERG_PUBLIC_HEADERS)

set(ICEBERG_PUBLIC_HEADER_CHECK_SOURCE
"// Generated from iceberg's installed public headers.\n")
foreach(ICEBERG_PUBLIC_HEADER IN LISTS ICEBERG_PUBLIC_HEADERS)
file(RELATIVE_PATH ICEBERG_PUBLIC_HEADER_RELATIVE_PATH "${ICEBERG_PUBLIC_INCLUDE_DIR}"
"${ICEBERG_PUBLIC_HEADER}")
string(APPEND ICEBERG_PUBLIC_HEADER_CHECK_SOURCE
"#include <${ICEBERG_PUBLIC_HEADER_RELATIVE_PATH}>\n")
endforeach()
string(APPEND ICEBERG_PUBLIC_HEADER_CHECK_SOURCE "\nint main() { return 0; }\n")

file(GENERATE
OUTPUT "${CMAKE_CURRENT_BINARY_DIR}/public_headers_check.cc"
CONTENT "${ICEBERG_PUBLIC_HEADER_CHECK_SOURCE}")

add_executable(public_headers_check "${CMAKE_CURRENT_BINARY_DIR}/public_headers_check.cc")
target_link_libraries(public_headers_check PRIVATE ${ICEBERG_BUNDLE_TARGET}
${ICEBERG_REST_TARGET})
14 changes: 13 additions & 1 deletion mkdocs/docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,14 @@

**Required:**

- C++23 compliant compiler (GCC 14+, Clang 18+, MSVC 2022+)
- C++23 compliant compiler (GCC 14+, Clang 18+, MSVC 2022+) to build iceberg-cpp itself
- CMake 3.25+
- [Ninja](https://ninja-build.org/) (recommended build backend)

**Using iceberg-cpp from your project:** the installed public headers require
C++20 at minimum, so applications that link against iceberg-cpp can be
compiled as C++20 or later. The library itself is still built as C++23.

## Quick Start

```bash
Expand Down Expand Up @@ -112,6 +116,14 @@ If using provided Apache Arrow, include both paths:
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH="/path/to/install;/path/to/arrow"
```

The examples build as C++20 by default, which is the minimum standard supported
by the public headers. Set `ICEBERG_EXAMPLE_CXX_STANDARD` to `23` to build them
as C++23 instead:

```bash
cmake -S . -B build -G Ninja -DCMAKE_PREFIX_PATH=/path/to/install -DICEBERG_EXAMPLE_CXX_STANDARD=23
```

## Customizing Dependency URLs

If you experience network issues when downloading dependencies, you can override the download URLs using environment variables:
Expand Down
2 changes: 1 addition & 1 deletion mkdocs/docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ iceberg-cpp is a C++ implementation of [Apache Iceberg™](https://iceberg.apach

## Key Features

- **Modern C++23** — Built with ranges, concepts, `std::expected`, and other modern idioms
- **Modern C++** — Built as C++23 with ranges, concepts, and other modern idioms; public headers require only C++20
- **Cross-Platform** — Builds and runs on Linux, macOS, and Windows
- **Spec Compliance** — Full table spec support today; Puffin, View, and UDF specs are on the roadmap
- **Arrow-Native** — Uses the Arrow C Data Interface as the primary data API
Expand Down
Loading
Loading