Unable to copy share point files through ADF

Question

I am facing issue now! I followed all the steps https://learn.microsoft.com/en-us/azure/data-factory/connector-sharepoint-online-list?tabs=data-factory and tried to read pdf files from sharepoint through ADF.

source: Sharepoint(PDF files)(I selected binary)

Target: ADL(binary)

ADF Activities:

1.) Web - it worked and i can able to get access token

2.) Copy - source as http connector to retrieve access token, sink as ADL (binary) and when i ran the pipeline I am getting following error

"Code": 22755, "Message": "ErrorCode=HttpRequestFailedWithClientError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Http request failed with client error, status code 400 BadRequest, please check your activity settings. If you configured a baseUrl that includes path, please make sure it ends with '/'.\nRequest URL: https://sharepoint.com/sites/site/_api/web/GetFileByServerRelativeUrl('/site/mysite/Shared Documents/sample.pdf')/$value.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=System.Net.WebException,Message=The remote server returned an error: (400) Bad Request.,Source=System,'", "EventType": 0, "Category": 5, "Data": {}, "MsgId": null, "ExceptionType": null, "Source": null, "StackTrace": null, "InnerEventInfos": [] } ],

Answer 1

Based on this old thread, the error code that you are having 22755 is related to ADF and it indicates a client-side failure, specifically a 400 Bad request error.

I think that the service principal or managed identity being used are missing the necessary permissions to access Azure Data Lake Storage Gen2.

Even if you have an access token for SharePoint, the service principal or managed identity might still lack the required permissions for Azure Data Lake Storage Gen2.

To troubleshoot this error, consider the following steps:

1. Review the Detailed Error Message: Check the specific error message from Azure Data Lake Storage Gen2. If it indicates a transient issue, try the operation again.
2. Verify Permissions: Ensure the service principal or managed identity has the necessary permissions for both SharePoint and Azure Data Lake Storage Gen2. Refer to the Azure documentation on service principal authentication for guidance.
3. Check Firewall Rules: Confirm that the firewall settings for Azure Data Lake Storage Gen2 allow Azure Data Factory IP addresses. See the Azure documentation on configuring Azure Storage firewalls and virtual networks for more details.
4. Validate Base URL Configuration: Ensure the base URL in your activity configuration ends with a forward slash (/). If the URL includes a path, it must end with a forward slash to prevent errors.

I hope this clarifies the error code and assists you in troubleshooting the issue. Feel free to ask if you have any more questions or concerns.