Відео

This is what people should call "small", anything below 1B! Thanks for your video. By the way, I played around with the quantized version, the result is unbelievably good! I shared a post on Twitter and mentioned you and shared the Colab. Take a look at it. I tried 8 bits and 4 bits. It's odd how 4 bits is almost the same as the base model!

I'd love to know how to do that with openwebui and a local model in a single GPU. Do we need to use FAISS or what RAG?

can i use it with vertex ai?

how to use it in Vertex-AI ?

Any idea how this works? I am trying to work out whether to use an index of relevant context or use the context cache feature. It seems like the details are a closely guarded secret which would mean the only way for me to decide between the two would be to test both lots. The use cases seem to be very similar Option 1 - Give google a bunch of context, hope that it's good and then run queries against it Option 2 - index my context and add information as needed using RAG The RAG approach would lead to more tokens being used but at least I know how it works so can set my expectations for how it works. The google approach would be cheaper but I don't know how the context has been processed. I cant intentionally format my data for optimal performance.

Thank you!

I'm enthusiastic about these smaller models. Thanks for covering this!

great video - thanks!

Eh tried "OCR with regions" using microsoft/Florence-2-large on a Firefox window, got all menus and all tabs grouped in a single region, with the captions OCRed with a lot of spelling errors. It's nice they released it, but it's not really useful as a screen reader out-of-the-box (so results similar to Tesseract). It could perhaps do better after finetuning...

Locate unmasked person in the image. Call corona police to arrest suspect.

It would be great if you can show a finetuning example!

Great, yes, fine tune would be very interesting.

I think fine-tuning for OCR would be a good demo. OCR in the real world with images of documents is much harder than OCR on electronic documents so would be cool to see how a small model like this does as an alternative to Claude/GPT4.

Please do fine-tuning for Object detection

Thanks, Sam! I always appreciate your videos. I would love your take on how Florence-2 compare with Apple's 4M-21.

Where is the dataset? I couldn't find the release

It's also good at OCR for hand written documents

Would be interested on how much memory is required to run these models. they seem pretty small even unquantized. Maybe I will try it later on my 8GB M1 Mini. One thing I am curious about: at 3:38 , the description for the image is wrong in ways that seem odd. The title is described as being on top with the "20 Years of ..." underneath and Ron's tie is described as red and hair blonde. I wonder if this is just vagaries of the model (placement data would be strange) or over reliance on training data. Or a straight up mistake in 'creating' the paper (which would probably be the most disturbing😉).

Thanks Sam!! Please keep up the great work...

awesome, thanks

Thanks for the great content. A video going through the fine-tuning process on this one would be amazing. I am not sure how this could scale to a video implementation (probably passing a frame each time).

Wow, those models are tiny. Have some engagement!

It is important to remember that there is a cost of $4.50 / 1 million tokens per hour (storage)

Hi Sam, thanks for the video. What do you think about how does it compare with Phi3-V? My take is that this is more raw and better for fine tuning, do you also think so?

Thanks for the information this is great. Can i fine tune it for certain specific images like few short learning. Can you put a tutorial for the same it will be great full.

Hi Sam. Thank you for the videos. I've been playing around with some of the smaller vision models and trying to implement batched inferencing with little success. If you were trying to accomplish running multiple VQA style questions against the same image quickly, how would you go about that goal? Is batching even in the right direction I should be looking?

We request you to do fune tuning on object detection. Because, all llms are useful generating text oupit only. Thanks in advance

Thanks for your work on sharing this information. Much easier to watch your content than keep my ear to the ground all day trying to keep up. Much appreciated, sir.

I wonder how much performance would be affected when something so distilled then gets quantized? Also, it seems amazing that it can handle segmentation for an unspecified set size! With Phi3 Vision you would need to provide a token to represent, say, each giraffe you want to identify.

Vqa tutorial would be nice!

Thank you - it looks interesting:)

I did try it, it is good and works for me, different from Microsoft edge TTS.

Thank you. This is saving me a loooooooooot of time

Not taking audio in is bizarre. any ideas why not?

can a retrieval QA chain work with the memory function?

can a retrieval chain work with memory function? I have been trying that for couple of days, but it doesn't work.

The only issue is that claude 3.5 doesnt have enough use without the pro plan to be good.

The thing I'm worried about, is that 3.5 is extremley censored, like wwwwwwaaaay to much.

Thank god possibly a standardized way to develop LLM Tool-Use 🎉

Seems unnecessary

Hi Sam. THX, I am a newbee. I don´t understand why I have to use google Colab?. Is there a difference if i use the AI Anthropic Opus directly? The Output is the same or not? I wil get the prompt with variables in google colab or direct in A. Opus.

Please provide any alternative solution for using this with open source models

Why people dont give Dash the love it deserves. Such powerful python UI maker

Great video. Thanks

Very useful. Thanks

Unuseable due to ridiculous message limits even on paid plan. Misleading

What about Taipy

This is super useful! ❤

i actually switched it it as my primary model, it's so much better

Claude 3.5 sonnet is just so good at coding! Something GPT-4o doesn't understand it gets it with minimal examples whereas GPT-4o is the opposite.