I'm working on a connector that needs to convert CSV files into Parquets, but I can't get Parquet.Net or ParquetSharp to work. Does anybody know why this might be and how I can get around it?
Hey Blake,
The reason why ParquetSharp fails is that ParquetSharp does not support x86 processes, which both SLScripting and SLAutomation are.
As for Parquet.Net, version 4.25.0 can work so long as you avoid certain compression styles as these will cause Parquet.Net to load a dependency that also does not support x86 processes. I recommend using Gzip compression as that is confirmed to be safe.
Also avoid naming columns with the JsonPropertyName Attribute as that can also cause that dependency to be loaded and cause issues.
These issues don't apply to GQI Ad Hoc DataSources as they run on a x64 process.
I hope this helps,
Bauti