Before the AWS Community Day second edition this year, we were discussing the next fun challenge we should build. We decided to create an interactive application based on quizzes and scores, nothing too fancy. Here’s how this eventually shaped out.
The Initial Concept
Imagine this: around 200 cloud enthusiasts gathered at AWS Community Day, exicted about the latest cloud news. We’d been part of this awesome event before, and this time, we wanted to bring something new to the table. Something fun and a bit competitive for the community.
Why not run a high-energy tech quiz? One that tests cloud knowledge and ramps up the excitement with every round.
And the prize? A new iPad for the champion.
We started by picturing a straightforward technical challenge. The core concepts were:
- Multiple technical quiz rounds focused on technical topics
- After each round, the bottom 50% of participants would be eliminated
- Notifications to participants regarding their score
Starting Simple: From Google Forms to Custom Solution
The Google Forms Approach
Why reinvent the wheel, we asked. Let’s use the tools we already have. So I suggested to use Google Forms. This way we avoid building a form system, we have built-in response collection in Google Sheets and it’s pretty easy to set up.
Ha, nice try.
A couple hours later, we had already run into limitations:
- We wanted to customize forms for each participant (with their name and info)
- We needed scoring logic based on both accuracy and response time
- We had to manipulate data in various ways, which was pretty challenging to achieve in Google Forms
- Connecting Google Forms to our notification system would be interesting, but no thanks
Let’s search for other alternatives then.
The Typeform Approach
We found this nice solution with many interesting integrations, very nice forms, and lots of customization options. This could be it. Let’s try a dummy form and see where we could send our answers.
Docs, Sheets, Excel, MySQL – yeah, this could be something. Let’s set up a DB then, I thought. However, doesn’t this defeat the purpose of “simple and nothing fancy”?
Well, ok. Then let’s build our own solution.
Execution flow
1. Participant scans a QR code
2. Fills registration form with phone number, name, email
3. Forms are created for each challenge, uploaded to S3, and pre-signed URLs are generated for retrieval
4. Data is stored in Aurora
5. Admin starts a challenge
6. A pre-signed URL is fetched from Aurora and sent to the Shorten Step
7. Using the participant data and short URL, the send-sms method sends out an SMS notification
8. Participant receives SMS, opens the short URL and views the form
9. When the HTML form is first opened, a timestamp is captured in JavaScript to track response time
10. A POST request delivers all the information to the FastAPI endpoint
11. The application pulls live Aurora data for real-time display
Building a Custom Form Solution
Once we took this decision, the sky was the limit. We had numerous ways of integrating all these services together, manipulate the data, build the forms and so on. So we’re going to need JUST a backend with a few API endpoints, a database, a notification system and some HTML forms. I am going to explain why and how we chose our tools for this solution. Let’s start with the beginning – HTML forms.
What’s the cheapest way to host some simple forms?
The S3 Form Arhitecture
1. HTML Templates
These templates contained placeholders that would be replaced with personalized information like the participant’s name, email, and unique identifier.
<!-- templates/challenge1_template.html -->
<div class="welcome-message">
<h2>Welcome, {{name}}!</h2>
<p>This is your personalized AWS quiz. Answer the questions to get points and climb the leaderboard!</p>
<p><strong>Your email:</strong> {{email}}</p>
</div>
Custom HTML Templates
2. Personalized Form Generation
When a participant registered, we generated customized forms for all challenges at once. Each form was tailored to the specific participant, with their information embedded directly in the HTML.
async def generate_all_challenge_forms(user_id, email, name):
"""
Generate personalized forms for all challenges for a user.
"""
try:
form_urls = {}
# Generate forms for all challenges
for challenge_num in CHALLENGES.keys():
form_url = await generate_personalized_form(user_id, email, name, challenge_num, skip_db_store=True)
if form_url:
form_urls[challenge_num] = form_url
else:
logger.error(f"Failed to generate form for challenge {challenge_num}, user {user_id}")
# Store all form URLs in database
success = database.store_multiple_form_links(user_id, form_urls)
logger.info(f"Generated {len(form_urls)} forms for user {user_id}")
return form_urls
except Exception as e:
logger.error(f"Error generating all challenge forms: {str(e)}")
return None
Form Generation Function in FastAPI
3. S3 Storage Strategy and Lifecycle Management
We developed a pretty solid, old-school S3 storage strategy that organized forms into distinct directories based on their status.
This organization made it easy to track participation status and prevent late submissions. And guess how the lifecycle management of the forms would work? When a participant submitted their form response, we would just move the object from forms to answered.
This way, nobody needed to actually login, we could keep track of everyone’s responses and prevent multiple submissions. The unanswered directory was dedicated to forms that were not submitted during a challenge. So if a challenge ended and there were some forms unsubmitted, we would automatically move those in the unanswered directory and the user won’t participate in the remaining challenges.
S3_BUCKET/
├── forms/ # Active, unsubmitted forms
│ ├── challenge1/
│ ├── challenge2/
│ └── challenge3/
├── answered/ # Completed forms
│ ├── challenge1/
│ ├── challenge2/
│ └── challenge3/
└── unanswered/ # Forms not submitted when challenge ended
├── challenge1/
├── challenge2/
└── challenge3/
S3 Structure for Storing Forms
4. Pre-signed URLs for Secure Access
A critical aspect of our design was using pre-signed URLs to grant temporary access to the forms.
This approach gave the following benefits and a major surprise. Keep reading.
- Security – forms weren’t publicly accessible
- Limited access – URLs expired after 24 hours; well, technically
- Simplified authentication – participants didn’t need to log in
- Participant tracking – each URL was unique and mapped to a participant/challenge combination
# Generate pre-signed URL that expires after 24 hours (86400 seconds)
form_url = s3_client.generate_presigned_url(
'get_object',
Params={
'Bucket': S3_BUCKET_NAME,
'Key': s3_path,
'ResponseContentDisposition': 'inline',
'ResponseContentType': 'text/html'
},
ExpiresIn=86400,
HttpMethod='GET'
)
S3 Pre-Signed URL Generation Function in FastAPI
5. URL Shortening for Form Links
The pre-signed S3 URLs were quite lengthy, making them impractical for SMS notifications. We implemented a URL shortening step.
async def shorten_form_url(url_long):
"""
Shortens a form URL using the CloudFront endpoint.
"""
try:
shortener_endpoint = "https://awsfn.com/admin_shrink_url"
payload = {
"url_long": url_long,
"cdn_prefix": "awsfn.com"
}
async with aiohttp.ClientSession() as session:
async with session.post(shortener_endpoint, json=payload) as response:
if response.status == 200:
result = await response.json()
if "url_short" in result:
return result["url_short"]
# Fallback to original URL if shortening fails
return url_long
except Exception as e:
logger.error(f"Error shortening URL: {e}")
return url_long
Shorten URL Function in FastAPI
This transformed unwieldy URLs like:
https://aws-community-day-forms.s3.amazonaws.com/forms/challenge1/user_42.html?AWSAccessKeyId=AKIAZQ3DQOKKHUNUMYDJ&Signature=6iPKUAqbywMdlGQR0cZ1bQQgnDc%3D&Expires=1742288401
Into concise links like:
https://awsfn.com/pszttib
This AWS Blog post and the CloudFormation stack saved us a few hours on the actual implementation.
We just deployed the stack and that’s it, we had our own private URL shortener service.
Optimization Challenges
1. Bulk Form Generation
Generating forms individually for each participant and challenge would have been too slow. We optimized by generating forms at register time for all challenges.
This approach frontloaded the generation work to registration time, when the system load was lighter, rather than when a challenge started and all participants would need forms simultaneously.
@app.post("/api/register")
async def register_user(
name: str = Form(...),
email: str = Form(...),
phone: str = Form(...),
company: str = Form(None)
):
# Register user in database
# ...
# Generate ALL forms for ALL challenges at registration time
form_urls = await generate_all_challenge_forms(user_id, email, name)
# Return success response with the first form URL
return {
"success": True,
"message": "Registration successful!",
"user_id": user_id,
"form_urls": form_urls,
"current_form_url": form_urls.get("1", "")
}
API Endpoint for user registration and Form Generation function in FastAPI
2. Database Storage of Form URLs
To avoid regenerating forms, we stored all form URLs in the database.
This allowed us to quickly retrieve form URLs when needed, such as when sending SMS notifications.
def store_multiple_form_links(user_id, form_urls):
"""
Store multiple form links for a user in the database.
"""
try:
conn = get_db_connection()
cursor = conn.cursor()
# Use a transaction for all inserts
cursor.execute("START TRANSACTION")
for challenge_num, form_url in form_urls.items():
cursor.execute(
"""
INSERT INTO form_links (user_id, challenge_num, form_url, created_at)
VALUES (%s, %s, %s, %s)
""",
(user_id, challenge_num, form_url, current_timestamp)
)
conn.commit()
logger.info(f"Stored {len(form_urls)} form links for user {user_id}")
return True
except Exception as e:
logger.error(f"Error storing form links: {e}")
if conn:
conn.rollback()
return False
finally:
if conn:
conn.close()
Function to store Pre-Signed URLs in the database in FastAPI
Outcome and Benefits
Our custom S3-based form system provided numerous advantages:
Complete Control & Personalization – we could customize every aspect of the form appearance and behavior
Response Time Tracking – we could measure how quickly each participant completed challenges
Lifecycle Management – forms moved through a clear process from active to answered or unanswered
Scalability – the system easily could handle hundreds of concurrent participants
Integration – forms directly submitted to our API, giving us the ability to play with the data as much as we wanted
While it obviously required more initial development effort than using Google Forms, the flexibility and control it provided were essential to what we wanted to achieve.
FastAPI: Shaping the Backend
After establishing our S3-based form system, we needed a robust and efficient backend to handle form submissions, manage the application state, and serve real-time updates. Yup, I said real-time because once we started to build our custom solution, we thought it would be much cooler to have an actual leaderboard updating in real-time.
So FastAPI came as the ideal choice for our application. Here’s why:
1. Asynchronous Support
FastAPI’s native async capabilities were crucial for handling concurrent submissions during challenge periods when dozens or hundreds of participants might submit answers simultaneously.
This was especially important for maintaining a responsive leaderboard that updated in real-time.
@app.post("/api/submit/{challenge_num}/{user_id}")
async def submit_challenge(challenge_num, user_id, score, answers):
# Process submission asynchronously
Async function in FastAPI
2. Built-in WebSocket Support
The real-time leaderboard was a cornerstone of our application. FastAPI’s native WebSocket support made this feature straightforward to implement.
This let us push leaderboard updates to all connected clients whenever new submissions arrived, creating an engaging, dynamic experience.
@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
await manager.connect(websocket)
# Send current leaderboard immediately
# Then keep connection open for updates
Web-Socket Endpoint in FastAPI
3. Background Task Processing
For time-consuming operations like sending SMS notifications to hundreds of participants, FastAPI’s background tasks were invaluable.
This prevented API requests from timing out during lengthy operations and improved the admin experience.
@app.post("/api/admin/start-challenge/{challenge_num}")
async def start_challenge(challenge_num, background_tasks: BackgroundTasks):
# Start sending SMS notifications in the background
background_tasks.add_task(
send_notifications_background,
qualified_users=qualified_users,
challenge_num=challenge_num
)
# Return immediately with success message
return {"success": True, "message": "Challenge started!"}
Task sent for processing in the background in FastAPI
Key API Features
1. Submission Processing
The form submission endpoint needed to:
- Validate incoming data and calculate response times (between form view and submission)
- Store answers and scores in the database and update the leaderboard
- Move forms from active to answered in S3
2. Challenge Management
For us, the event administrators, we created endpoints to control the challenge flow. These endpoints orchestrated the entire event lifecycle, from initializing challenges to processing eliminations and declaring winners.
# Start a challenge and send notifications
@app.post("/api/admin/start-challenge/{challenge_num}")
# End a challenge and process eliminations
@app.post("/api/admin/end-challenge/{challenge_num}")
API Endpoints for challenge controls in FastAPI
3. Real-time Progress Tracking
For long-running operations like sending SMS notifications, we implemented progress tracking.
This allowed us to display real-time progress bars, providing visibility into background operations.
@app.get("/api/admin/sms-progress/{challenge_num}")
async def get_sms_progress_api(challenge_num: str):
progress = get_sms_progress(challenge_num)
return {
"status": progress["status"],
"total": progress["total"],
"sent": progress["sent"],
"failed": progress["failed"],
"progress_percent": progress["progress_percent"]
}
Progress Tracking function in FastAPI
Challenges and Solutions
For long-running operations like sending SMS notifications, we implemented progress tracking.
This allowed us to display real-time progress bars, providing visibility into background operations.
1. Elimination logic
After each challenge, we needed to eliminate the bottom 50% of participants. This required some fuzzy calculations like:
- Sorting participants by score and response time
- Identifying the bottom half for elimination
- Updating their status in the database
- Sending elimination notifications
We implemented a sorting algorithm that considered both score accuracy and response time as tiebreakers.
2. Form Lifecycle Management
Moving forms between S3 directories required coordination between the API and storage operations.
This approach prevented late submissions after challenges closed.
# When a form is submitted
s3_client.copy_object(
Bucket=S3_BUCKET_NAME,
CopySource={'Bucket': S3_BUCKET_NAME, 'Key': source_key},
Key=f"answered/challenge{challenge_num}/user_{user_id}.html"
)
s3_client.delete_object(Bucket=S3_BUCKET_NAME, Key=source_key)
# When a challenge ends, move all remaining forms
for form in forms:
# Move from forms/ to unanswered/
Function for copying objects from a S3 directory to another in FastAPI
3. The Admin Dashboard
The admin dashboard provided us with real-live data statistics, challenge control, status for notifications, list of eliminated participants and winner announcement controls.
The solution so far
The combination of the S3-based form system with the FastAPI backend gave us a powerful, integrated platform that transformed a simple quiz into an engaging, competitive event experience.
Aurora Serverless: Powering the Database Layer
While S3 hosted our forms and FastAPI handled our backend logic, Aurora served as the robust database backbone that tied everything together. In the following section, we’ll discuss why we leveraged this powerful managed database service in our solution.
Our application required a reliable, scalable database that could handle concurrent operations during peak submission periods. Aurora MySQL Serverless offered several advantages that made it ideal for our needs.
1. Managed Service Benefits
As a DevOps-focused team, we appreciated Aurora’s fully managed nature. We didn’t have to worry about:
- Scaling on peak traffic
- Database server provisioning
- Backups and recovery
- High availability configuration
This allowed us to focus on application development rather than database administration.
2. MySQL Compatibility
Aurora’s MySQL compatibility meant we could use familiar tools and libraries. Our team was already comfortable with MySQL, making Aurora a natural choice. We used the PyMySQL library for database interactions.
Database Schema Evolution
Our schema evolved as the application requirements grew.
We maintained separate users and participants tables. The users table stored complete registration information, while the leaner participants table was used for leaderboard queries to improve performance.
Here’s how some of our key tables were structured.
CREATE TABLE users (
id INT AUTO_INCREMENT PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL,
name VARCHAR(255) NOT NULL,
phone VARCHAR(50) NOT NULL,
company VARCHAR(255),
registered_at DATETIME NOT NULL
);
CREATE TABLE participants (
id INT AUTO_INCREMENT PRIMARY KEY,
email VARCHAR(255) UNIQUE NOT NULL
);
Part of the FastAPI database initialization script
Key Database Operations
1. Form Submission Processing
When a participant submitted answers, we needed to record their score and update the leaderboard:
# Calculate response time
cursor.execute(
"SELECT first_viewed_at FROM form_views WHERE user_id = %s AND challenge_num = %s",
(user_id, challenge_num)
)
view_data = cursor.fetchone()
response_time_seconds = (submit_time - view_data["first_viewed_at"]).total_seconds()
# Save answers and score
cursor.execute(
"""
INSERT INTO user_answers (user_id, challenge_number, answers, score, submitted_at)
VALUES (%s, %s, %s, %s, %s)
""",
(user_id, int(challenge_num), answers, score, datetime.now())
)
# Update challenge score with response time
cursor.execute(
"""
REPLACE INTO challenge_scores
(participant_id, challenge_number, score, timestamp, response_time_seconds)
VALUES (%s, %s, %s, %s, %s)
""",
(participant_id, int(challenge_num), score, datetime.now(), response_time_seconds)
)
Various queries executed sequentially once an answer was submitted
2. Leaderboard Generation
Generating the leaderboard required joining multiple tables and some complex sorting:
def get_db_leaderboard_for_challenge(challenge_num):
"""Get leaderboard data for a specific challenge."""
try:
# Get elimination data
cursor.execute("""
SELECT p.email, e.challenge_number
FROM eliminated e
JOIN participants p ON e.participant_id = p.id
""")
eliminated_map = {row["email"]: row["challenge_number"] for row in cursor.fetchall()}
# Get participants for this challenge
cursor.execute("""
SELECT
p.id AS participant_id,
p.email,
u.name,
cs.score AS challenge_score,
cs.timestamp,
cs.response_time_seconds
FROM
participants p
JOIN
users u ON p.email = u.email
LEFT JOIN
challenge_scores cs ON p.id = cs.participant_id AND cs.challenge_number = %s
""", [int(challenge_num)])
# Process results into leaderboard format
leaderboard = []
for row in results:
# Add to leaderboard with elimination status
is_eliminated = row["email"] in eliminated_map
leaderboard.append({
"rank": 0, # Will be assigned after sorting
"email": row["email"],
"name": row["name"],
"score": row["challenge_score"] or 0,
"timestamp": row["timestamp"],
"eliminated": is_eliminated,
"elimination_challenge": eliminated_map.get(row["email"]),
"sort_total_time": row["response_time_seconds"] or 999999
})
# Sort by score and response time
leaderboard.sort(key=lambda x: (
1 if x["eliminated"] else 0, # Eliminated participants at the end
-x["score"], # Higher scores first
x["sort_total_time"] # Faster times first
))
# Assign ranks
rank = 1
for entry in leaderboard:
if not entry["eliminated"]:
entry["rank"] = rank
rank += 1
else:
entry["rank"] = "-"
return leaderboard
except Exception as e:
logger.error(f"Error getting leaderboard: {e}")
return []
Function that retrieves the contest participants and processes the leadboard sorting for a given challenge
3. Challenge Qualification
Between challenges, we needed to determine which participants qualified for the next round:
def transfer_qualified_participants(from_challenge, to_challenge):
"""Transfer qualified participants from one challenge to another."""
try:
# Find qualified participants from the previous challenge
query = """
SELECT p.id, p.email
FROM participants p
JOIN challenge_scores cs ON p.id = cs.participant_id
LEFT JOIN eliminated e ON p.id = e.participant_id AND e.challenge_number = %s
WHERE
cs.challenge_number = %s AND
e.participant_id IS NULL
"""
cursor.execute(query, (int(from_challenge), int(from_challenge)))
qualified_participants = cursor.fetchall()
# Insert these participants for the new challenge
for participant in qualified_participants:
participant_id = participant['id']
# Insert default score for new challenge
insert_query = """
INSERT IGNORE INTO challenge_scores
(participant_id, challenge_number, score, timestamp)
VALUES (%s, %s, 0, %s)
"""
cursor.execute(
insert_query,
(participant_id, int(to_challenge), datetime.now())
)
return True
except Exception as e:
logger.error(f"Error transferring participants: {e}")
return False
Function that transfers participants from a challenge to the next one
Enforcing the application
Aurora gave us a solid foundation for our solution’s database layer. Its combination of MySQL compatibility, managed service benefits, and performance characteristics made it the perfect choice for our needs.
By leveraging Aurora alongside S3 for form storage and FastAPI for application logic, we created a well-integrated system that provided an interesting, fun, and enjoyable experience for the participants. And for us, it was about monitoring it effectively while ensuring it scales reliably and stays bulletproof.
Infrastructure and DevOps: Foundation of the application
As a team with extensive DevOps experience, we knew that creating a solid infrastructure foundation would be very important to our application’s success. Rather than treating infrastructure as an afterthought, we made it a priority from day one, enabling rapid development iterations and faster deployments.
We approached our AWS environment with a methodical, layered strategy that allowed us to move quickly without sacrificing stability or security.
Layer-Based Organization
1. Shared Layer
- Networking (VPC, subnets, security groups)
- DNS and certificate management
- Common security controls (KMS, IAM)
- Shared storage resources
3. Application Layer
- ECS Fargate cluster for the FastAPI application
- Load balancer and target group
- CloudFront distributions for content delivery
- Auto-scaling configurations
2. Stateful Layer
- Aurora MySQL Serverless
- S3 buckets for form storage
4. CI/CD Layer
- CodePipeline and CodeBuild resources
- ECR repositories for container images
- Deployment IAM roles and policies
This layered approach meant we could establish our core infrastructure quickly, then iterate on specific components without disrupting the entire stack.
The surprise
We were almost done with challenge 2, so we decided it’s time to move to challenge 3. So in proceed we ended challenge 2, the backend logic automatically eliminated the half bottom participants (sorry guys), and then the leaderboard was now updated with the remaining finalists for challenge 3.
We started challenge 3, SMS notifications begin to flow to the last remaining participants, everything was working great. In a matter of seconds, people started showing up at our booth and showing us their phones. Each of their phones displayed something like:
Oh boy, what could this be? We started brainstorming this ad hoc, taking it step by step from the beginning. Like we usually do in our workflows. Something broke our S3 pre-signed URLs.
We did a bit of research and found, after a while, this resource.
Our S3 generate_presigned_url() call was made by the ECS task role where FastAPI was running:
# Generate pre-signed URL that expires after 24 hours - NOT SO FAST!
form_url = s3_client.generate_presigned_url(
'get_object',
Params={
'Bucket': S3_BUCKET_NAME,
'Key': s3_path,
'ResponseContentDisposition': 'inline',
'ResponseContentType': 'text/html'
},
ExpiresIn=86400,
HttpMethod='GET'
)
Typical boto3 call for generation of pre-signed URLs
But, as we are continuously learning from AWS Documentation, a pre-signed URL generated by an IAM instance profile is valid up to 6 hours.
So guess how long did it take from the generation of the forms until starting challenge 3? A few minutes over six hours. Yep. A few minutes.
The solution
After we found the issue, we needed to come up with a plan: we need fresh pre-signed URLs. We started by checking CloudWatch logs (I was exporting every bit of information there, good call) and we quickly found the participants for challenge 3 among other logs.
We exported a CSV file with the latest logs, used Claude to set up a Python script that will process the big CSV into a friendly .txt file with only the relevant fields for us.
Prepared a VPN connection to Aurora (got that covered from the beginning as well), so we were able to execute queries.
Finally, we came up with another script that should fetch the remaining users from the .txt file, and for each user should re-generate the pre-signed URL, then would update the database with the new value.
I ran it locally watching the terminal processing each user one by one. So, the URLs are fresh now. Let’s check. We’ve used a test user to quickly determine if the new pre-signed URLs are working, looked good. We restarted challenge 3, SMS notifications started to flow again and new forms submission started to arrive. Now we were relieved.
Another day at the office, another debug in production.
Conclusion
This project proved to us that sometimes the simple path isn’t always the right one. Building everything from the scratch gave us full control, real-time performance, and the flexibility to shape the experience exactly how we wanted. And yes, it came with an unforgettable debugging experience in front of 200 cloud enthusiasts. Looking back, it was fun. (Really.)
And the iPad went to the champion who combined strong AWS knowledge with lightning-fast reponses, congratulations again!
Meanwhile, we’re already missing Timișoara. But we’re grateful to return with even more experience than before, and we’re already looking forward to next year’s challenges.
