Split PDF files into individual slide images. Requires poppler-utils installed (brew install poppler on macOS or apt-get install poppler-utils on Ubuntu).
Split a PDF file into individual slide images using poppler’s pdftoppm.
For example, you can split the NewFrontiersInIR.pdf file into individual slide images:
Created 65 slide images in slides_output/I often want to gather context from a set of web pages.
Gather contents from URLs.
Get a website as md with Jina.
For example, these are what I might use as context for annotated posts
<examples>
<example-1>
Title:
URL Source: https://raw.githubusercontent.com/hamelsmu/hamel-site/refs/heads/master/notes/llm/evals/inspect.qmd
Markdown Content:
---
title: "Inspect AI, An OSS Python Library For LLM Evals"
description: "A look at Inspect AI with its creator, JJ Allaire."
author: ["Hamel Husain"]
date: 2025-06-23
order: 8
image: inspect_images/inspect_cover.png
---
A few weeks ago, I had the pleasure of hosting [JJ Allaire](https://en.wikipedia.org/wiki/Joseph_J._Allaire) for a110419--------------------------------------------------------------------------- ServerError Traceback (most recent call last) Cell In[9], line 1 ----> 1 _o = outline_slides('NewFrontiersInIR.pdf') 2 print(_o[:300]) Cell In[8], line 3, in outline_slides(slide_path) 2 def outline_slides(slide_path): ----> 3 return gem("Provide a numbered list of each slide with a one sentence summary of each. Just a numbered list please, no other asides or meta explanations of the task are required.", slide_path) File ~/git/prompts/hamel/hamel/gem.py:220, in gem(prompt, o, model, thinking, search) 218 contents = _content_payload(prompt, attachments, parts) 219 cfg = _build_config(thinking, search, parts) --> 220 resp = _generate_content(model, contents, cfg) 221 return resp.text File ~/git/prompts/hamel/hamel/gem.py:206, in _generate_content(model, contents, cfg) 204 def _generate_content(model, contents, cfg): 205 with _client() as client: --> 206 return client.models.generate_content(model=model, contents=contents, config=cfg) File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/google/genai/models.py:5230, in Models.generate_content(self, model, contents, config) 5228 while remaining_remote_calls_afc > 0: 5229 i += 1 -> 5230 response = self._generate_content( 5231 model=model, contents=contents, config=parsed_config 5232 ) 5234 function_map = _extra_utils.get_function_map(parsed_config) 5235 if not function_map: File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/google/genai/models.py:4012, in Models._generate_content(self, model, contents, config) 4009 request_dict = _common.convert_to_dict(request_dict) 4010 request_dict = _common.encode_unserializable_types(request_dict) -> 4012 response = self._api_client.request( 4013 'post', path, request_dict, http_options 4014 ) 4016 if config is not None and getattr( 4017 config, 'should_return_http_response', None 4018 ): 4019 return_value = types.GenerateContentResponse(sdk_http_response=response) File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/google/genai/_api_client.py:1388, in BaseApiClient.request(self, http_method, path, request_dict, http_options) 1378 def request( 1379 self, 1380 http_method: str, (...) 1383 http_options: Optional[HttpOptionsOrDict] = None, 1384 ) -> SdkHttpResponse: 1385 http_request = self._build_request( 1386 http_method, path, request_dict, http_options 1387 ) -> 1388 response = self._request(http_request, http_options, stream=False) 1389 response_body = ( 1390 response.response_stream[0] if response.response_stream else '' 1391 ) 1392 return SdkHttpResponse(headers=response.headers, body=response_body) File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/google/genai/_api_client.py:1224, in BaseApiClient._request(self, http_request, http_options, stream) 1221 retry = tenacity.Retrying(**retry_kwargs) 1222 return retry(self._request_once, http_request, stream) # type: ignore[no-any-return] -> 1224 return self._retry(self._request_once, http_request, stream) File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/tenacity/__init__.py:477, in Retrying.__call__(self, fn, *args, **kwargs) 475 retry_state = RetryCallState(retry_object=self, fn=fn, args=args, kwargs=kwargs) 476 while True: --> 477 do = self.iter(retry_state=retry_state) 478 if isinstance(do, DoAttempt): 479 try: File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/tenacity/__init__.py:378, in BaseRetrying.iter(self, retry_state) 376 result = None 377 for action in self.iter_state.actions: --> 378 result = action(retry_state) 379 return result File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/tenacity/__init__.py:420, in BaseRetrying._post_stop_check_actions.<locals>.exc_check(rs) 418 retry_exc = self.retry_error_cls(fut) 419 if self.reraise: --> 420 raise retry_exc.reraise() 421 raise retry_exc from fut.exception() File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/tenacity/__init__.py:187, in RetryError.reraise(self) 185 def reraise(self) -> t.NoReturn: 186 if self.last_attempt.failed: --> 187 raise self.last_attempt.result() 188 raise self File /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py:451, in Future.result(self, timeout) 449 raise CancelledError() 450 elif self._state == FINISHED: --> 451 return self.__get_result() 453 self._condition.wait(timeout) 455 if self._state in [CANCELLED, CANCELLED_AND_NOTIFIED]: File /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/concurrent/futures/_base.py:403, in Future.__get_result(self) 401 if self._exception: 402 try: --> 403 raise self._exception 404 finally: 405 # Break a reference cycle with the exception in self._exception 406 self = None File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/tenacity/__init__.py:480, in Retrying.__call__(self, fn, *args, **kwargs) 478 if isinstance(do, DoAttempt): 479 try: --> 480 result = fn(*args, **kwargs) 481 except BaseException: # noqa: B902 482 retry_state.set_exception(sys.exc_info()) # type: ignore[arg-type] File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/google/genai/_api_client.py:1201, in BaseApiClient._request_once(self, http_request, stream) 1193 else: 1194 response = self._httpx_client.request( 1195 method=http_request.method, 1196 url=http_request.url, (...) 1199 timeout=http_request.timeout, 1200 ) -> 1201 errors.APIError.raise_for_response(response) 1202 return HttpResponse( 1203 response.headers, response if stream else [response.text] 1204 ) File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/google/genai/errors.py:121, in APIError.raise_for_response(cls, response) 118 else: 119 response_json = response.body_segments[0].get('error', {}) --> 121 cls.raise_error(response.status_code, response_json, response) File ~/git/prompts/hamel/.venv/lib/python3.10/site-packages/google/genai/errors.py:148, in APIError.raise_error(cls, status_code, response_json, response) 146 raise ClientError(status_code, response_json, response) 147 elif 500 <= status_code < 600: --> 148 raise ServerError(status_code, response_json, response) 149 else: 150 raise cls(status_code, response_json, response) ServerError: 503 UNAVAILABLE. {'error': {'code': 503, 'message': 'The model is overloaded. Please try again later.', 'status': 'UNAVAILABLE'}}
def generate_annotated_talk_post(slide_path, video_source, # YouTube link or local MP4 path image_dir, transcript_path=None, example_urls=_annotated_post_urls, user_prompt=None): “Assemble the prompt for the annotated post. Optional user_prompt provides additional context/examples.”
# Check if video_source is a local MP4 or YouTube URL
is_local_video = Path(video_source).exists() and Path(video_source).suffix.lower() == '.mp4'
video_chapters = yt.yt_chapters(video_source)
slide_outline = outline_slides(slide_path)
transcript = Path(transcript_path).read_text() if transcript_path else yt.transcribe(video_source)
examples = gather_urls(example_urls)
_ = pdf2imgs(slide_path, output_dir=image_dir)
# Adjust prompt based on whether it's a YouTube video or local MP4
if is_local_video:
video_reference = f"the local video file {video_source}"
timestamp_note = "Note: For local MP4 files, timestamps cannot be linked directly. Just provide the timestamp in [MM:SS] format."
else:
video_reference = f"the YouTube video at {video_source}"
timestamp_note = f"Additionally, reference the correct timestamp in the form of a timestamped linked to the youtube video that corresponds to the start of each slide. The link to this presentation is {video_source} (so use this when adding timestamps please)."
prompt=f"""Attached is the transcript (in <transcript> tags) of a technical talk for the attached slides. I'd like to make an annotated presentation blog post as illustratd in <example-posts> tags.For each slide, provide a detailed synopsis of the information to maximize understanding for the reader for the purposes of educating the reader. Each section should provide enough commentary and info to understand the full context of that particular slide. The idea is that the reader will not have to watch the video and can instead read the material so the writing + slide should stand alone. Do not simply repeat the information on each slide, briefly describe what the slide is about, and capture supplementary information that was provided in the talk that is NOT in the slides. Be thoroughly detailed and capture useful asides or commentary as well, such that the notes you generate should be a legitimate value add on top of the slides.
When writing the article, provide markdown placeholders with appropriate captions where the slides will go. For example, you might have placeholder like this.
Note that images for this post will be placed in {image_dir}/
Refer to slides with naming convention (slide_1.png, slide_2.png, etc)
{timestamp_note}
I have included other annotated posts as an example for you to understand the format. These examples are in
Finally, there might be Q&A section of the talk that will not correspond to any slides at all. If that exists, list all those questions with answers in a Q&A section. If there is a Q&A section, it should be drafted to maximize learning such that people who have listened to the talk can understand the full context. Add timestamps if possible to each question in the Q&A as well. The post should be written from the perspective of Hamel Husain (me) who hosted the talk as part of a course on LLM Evals (https://bit.ly/evals-ai). Put a CTA at the beginning and end of the post in a tasteful way that is appropriate for a developer blog that looks something like the example posts, particularly following p1-intro.md.
Example CTA: We are teaching our last and final cohort of our AI Evals course next month (we have to get back to building). Here is a 35% discount code for readers.
Here is the transcript
Incase it is helpful, here is here is the video description with chapters from the talk. However, please use timestamps from the transcript when possible when constructing timestamped links.
Below is a brief slide outline (in addition to the attached pdf)
Here are example posts that I have previously written: {examples}
When writing the introduction, annotation and Q&A keep the following writing guidelines in mind:
Please go ahead and draft the post. Please also include front matter similar to the front matter in the examples and select the best slide from the talk as the cover image (which is not the title slide, but instead another interesting slide that is punchy). ““” if user_prompt: prompt += f”context/instructions from the user:” # Use the appropriate video source attachment = [slide_path, video_source] if is_local_video else [slide_path, video_source] draft_post = gem(prompt, attachment, model=‘gemini-3-pro-preview’) return draft_post
def generate_annotated_talk_post(
slide_path, video_source, # YouTube link or local MP4 path
image_dir, transcript_path:NoneType=None,
example_urls:L=['https://raw.githubusercontent.com/hamelsmu/hamel-site/refs/heads/master/notes/llm/evals/inspect.qmdhttps://raw.githubusercontent.com/hamelsmu/hamel-site/refs/heads/master/notes/llm/rag/p1-intro.md', 'https://raw.githubusercontent.com/hamelsmu/hamel-site/refs/heads/master/notes/llm/rag/p2-evals.md', 'https://raw.githubusercontent.com/hamelsmu/hamel-site/refs/heads/master/notes/llm/rag/p3_reasoning.qmd', 'https://raw.githubusercontent.com/hamelsmu/hamel-site/refs/heads/master/notes/llm/rag/p4_late_interaction.qmd', 'https://raw.githubusercontent.com/hamelsmu/hamel-site/refs/heads/master/notes/llm/rag/p5_map.qmd']
):
Assemble the prompt for the annotated post.