Skip to content

Support DataContent and UriContent (images) in GrokChatClient messages#76

Merged
kzu merged 2 commits intomainfrom
copilot/fix-grokchatclient-image-sending
Feb 20, 2026
Merged

Support DataContent and UriContent (images) in GrokChatClient messages#76
kzu merged 2 commits intomainfrom
copilot/fix-grokchatclient-image-sending

Conversation

Copy link
Contributor

Copilot AI commented Feb 20, 2026

GrokChatClient.MapToRequest silently dropped DataContent and UriContent from chat messages — images were never sent to the API.

Changes

  • GrokChatClient.cs: Added two new branches in the content-mapping loop:
    • DataContent with image media type → ImageUrlContent with data:<mediaType>;base64,<data> URI
    • UriContent with image media type → ImageUrlContent with the URI string

Both map to the proto's Content.image_url field, which accepts both URL and base64 data URIs.

Usage

var image = new DataContent(File.ReadAllBytes("photo.png"), "image/png");
var message = new ChatMessage(ChatRole.User, [new TextContent("What's in this image?"), image]);

// UriContent also works
var imageUrl = new UriContent(new Uri("https://example.com/photo.jpg"), "image/jpeg");
Original prompt

This section details on the original issue you should resolve

<issue_title>GrokChatClient fails to send images (DataContent)</issue_title>
<issue_description>## Describe the Bug

Adding DataContent (such as image) to chat message does not actually get appended to Grok request

Steps to Reproduce

    [SecretsFact("XAI_API_KEY")]
    public async Task GrokImage()
    {
        var messages = new List<ChatMessage>();
        var image = new DataContent(File.ReadAllBytes(@"C:\temp\xai.png"), "image/png");
        var chatMessage = new ChatMessage(ChatRole.User, [new TextContent("Tell me what you see in the image"),image]);
        messages.Add(chatMessage);

        var chat = new GrokClient(Configuration["XAI_API_KEY"]!).AsIChatClient("grok-4")
            .AsBuilder()
            .UseLogging(output.AsLoggerFactory())
            .Build();

        var options = new GrokChatOptions
        {
            ModelId = "grok-4-fast-non-reasoning",
            AdditionalProperties = new()
            {
            }
        };

        var response = await chat.GetResponseAsync(messages, options);
        output.WriteLine(response.Messages.Select(x => x.Text).Last());
    }

Expected Behavior

Grok detects the image

Exception with Stack Trace

Put the exception with stack trace here.

Version Info

Latest source code / v1.0 on nuget both affected

Additional Info

Adding something like this to GrokChatClient seems to fix it, though this is probably more to it.

                if (content is DataContent dataContent)
                {
                    gmsg.Content.Add(new Content { ImageUrl = new ImageUrlContent() { ImageUrl = $"data:{dataContent.MediaType};base64,{dataContent.Base64Data}" }});
                }

Back this issue
Back this issue

<agent_instructions>Look up the xai official docs on how attachments can be sent and make sure we do this in the right way, supporting both URL and data content being submitted.</agent_instructions>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

@kzu
Copy link
Member

kzu commented Feb 20, 2026

24 passed 24 passed 23 skipped

🧪 Details on Ubuntu 24.04.3 LTS

from retest v1.1.0 on .NET 10.0.3 with 💜 by @devlooped

…eUrlContent

Co-authored-by: kzu <169707+kzu@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix GrokChatClient to send images in messages Support DataContent and UriContent (images) in GrokChatClient messages Feb 20, 2026
@kzu kzu marked this pull request as ready for review February 20, 2026 10:47
Copilot AI requested a review from kzu February 20, 2026 10:47
@kzu kzu merged commit 956d133 into main Feb 20, 2026
4 checks passed
@kzu kzu deleted the copilot/fix-grokchatclient-image-sending branch February 20, 2026 10:49
@kzu kzu added the enhancement New feature or request label Feb 20, 2026
@devlooped devlooped locked and limited conversation to collaborators Mar 23, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GrokChatClient fails to send images (DataContent)

2 participants