FEAT: Bulk Copy #73
Open
FEAT: Bulk Copy #73
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a problem hiding this comment.
Pull Request Overview
This PR introduces new Bulk Copy functionality in the mssql_python library by implementing a new BCPClient class and enhancing BCPOptions and ColumnFormat to support flexible bulk copy operations. Key changes include:
- Addition of the BCPClient class and associated pybind11 bindings in C++ for managing BCP operations.
- Updates to BCPOptions and ColumnFormat with extra validations and support for new configuration parameters.
- New integration scripts and sample data/format files to facilitate testing of bulk copy operations.
Reviewed Changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/test_007_bcpOptions.py | Unit tests to validate the new BCPOptions and ColumnFormat behavior. |
| mssql_python/pybind/ddbc_bindings.{h,cpp} | Updated function pointer loading and inclusion of new BCP API pointers. |
| mssql_python/pybind/connection/connection.cpp | BCP-specific connection attribute handling added. |
| mssql_python/pybind/bcp/{bcp_wrapper.h,cpp} | New BCPWrapper implementation with methods for initializing and executing BCP. |
| mssql_python/CMakeLists.txt | Updated to include BCPWrapper sources and header directories. |
| mssql_python/constants.py | Added new enum for BCP control options. |
| mssql_python/bcp_options.py | Enhanced validation and options for bulk copy configurations. |
| mssql_python/bcp_main.py | New BCPClient class implementation that wraps BCPWrapper operations. |
| main_bcp.py | Standalone example script demonstrating the BCP functionality. |
| Data and format files | Added sample BCP data (.bcp), Unicode data file, and format files (.fmt). |
Comments suppressed due to low confidence (1)
mssql_python/pybind/bcp/bcp_wrapper.cpp:197
- The two overloads of bcp_control show inconsistent error handling—one throws an exception when in an invalid state while the other returns SQL_ERROR. Consider adopting a consistent error handling strategy across both overloads.
if (!_bcp_initialized || _bcp_finished) {
| #include <stdexcept> | ||
| #include <unordered_map> | ||
| #include <string> // For std::to_string, std::string | ||
| #include <iostream> // Added for std::cout |
There was a problem hiding this comment.
[nitpick] Consider replacing raw std::cout debug prints with a dedicated logging framework to ensure consistent logging behavior in production code.
Copilot uses AI. Check for mistakes.
added 6 commits
June 12, 2025 10:21
… jahnvi/bcp_py_apis
… jahnvi/bcp_py_apis
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant enhancements to the Bulk Copy Program (BCP) functionality in the
mssql_pythonlibrary, including the addition of new classes, methods, and configuration options to support bulk data operations. It also includes updates to data files and format files to facilitate testing and integration.Enhancements to BCP functionality:
New
BCPClientclass for bulk copy operations:BCPClientinmssql_python/bcp_main.pyto manage bulk copy operations using the BCP utility. This includes initialization, validation, and execution of BCP operations with detailed logging.Updates to
BCPOptionsandColumnFormatclasses:ColumnFormatwith new attributes such asuser_data_typeandcol_name, along with default values forprefix_lenanddata_len.in,out,format,query) and file modes (native,char,unicode) inBCPOptions.Integration and usage:
New
main_bcp.pyscript:Initialization of BCP components:
mssql_python/__init__.pyto include imports forBCPClient,BCPOptions, andColumnFormat.Data and format file updates:
Employee data file (
EmployeeFullNames.bcp):Format files:
EmployeeFullNames.fmtandTestBCP.fmtto define data structure and encoding for bulk operations. [1] [2]Unicode data file (
data_unicode.csv):