NutriGram: How I Built a Privacy-First Multi-Assistant Nutrition Tracker Inside Telegram

Track your meals, not your data.

Meet NutriGram – a.k.a. “Macro‑Bot” – a Telegram mini‑app that turns a quick chat, photo, or voice note into a full macronutrient read‑out in seconds. It’s powered by a swarm of specialized OpenAI Assistants, runs on a lean Node.js backend, and never stores a single piece of personally‑identifiable information. You can start logging today at @self_nutri_bot.

1. Why build nutrition tracking inside Telegram?

Most food‑logging apps lose people at the install‑sign‑up‑onboard gauntlet. Telegram already lives on your phone, provides text/voice/image input for free, and – thanks to Telegram Stars – now offers in‑chat payments that bypass app‑store friction. That combination let me ship a fully‑featured tracker without designing another mobile UI or storing sensitive user profiles.

Privacy by design

• No email, phone number, or name is required – the bot receives a numeric chat ID only.
• Photos and audio are deleted right after they’re processed.
• Meal logs live for as long as you choose; delete a message in Telegram and it vanishes from the database as well.

2. Key features at a glance

Pricing sits at 100 Stars / month (≈ €3) with a 15‑day free trial and a discounted yearly plan.

3. Under the hood: a multi‑assistant AI system

Instead of one monolithic model, NutriGram chains seven purpose‑built OpenAI Assistants – each a specialist with its own system prompt and tools.

flowchart TD
    A[Telegram message] --> B{Handler Factory}
    B -->|text| T(TextHandler)
    B -->|photo| P(PhotoHandler)
    B -->|audio| A1(AudioHandler)
    P --> C(Image Type Check)
    C -->|food photo| D(Dish Description)
    C -->|label| L(Label Extraction)
    T --> I(Text→Ingredients)
    A1 --> STT(Audio Transcription)
    subgraph "Macro Assistants"
        D & L & I & STT --> M(Macro Calculator)
    end
    M --> DB[(MongoDB)]
    M --> TG[Telegram reply]

Assistant roster

1. Image Type Check – routes photos to label‑vs‑dish paths.
2. Nutrition Label Extractor – OCR + structuring of nutrition panels.
3. Dish Description Bot – turns a food photo into a natural‑language ingredient list.
4. Text → Ingredients – parses free‑form meal descriptions.
5. Audio Transcription – converts voice notes to text via Whisper.
6. Macro Assistant – looks up USDA food vectors and computes calories & macros.
7. False Input Check – filters spam or policy violations.

All assistants share a single thread ID per user session so context flows smoothly down the chain.

4. A step‑by‑step example: processing a food photo

1. User → sends a plate snapshot.
2. PhotoHandler uploads the image to OpenAI.
3. Image Type Check decides it’s a dish, not a label.
4. False Input Check approves the content.
5. Dish Description Bot returns: “A bowl of oatmeal topped with ½ banana, 10 g chia seeds, and a drizzle of honey.”
6. Macro Assistant converts that description into precise grams, queries the USDA embedding index, and sums the macros.
7. MongoDB stores the entry; Telegram replies with a formatted nutrient card and cumulative daily totals.

Total latency: ≈ 4.8 s end‑to‑end on a standard GPT‑4o model mix.

5. Tech stack & code snippets

Thread management pattern

// Create a thread once per chat session
const { id: threadId } = await openai.createThread();

// Re‑use across assistants
await imageTypeCheck.process(openai, threadId, imageUrl);
await dishDescriptionBot.process(openai, threadId, photoAnalysis);
await macroAssistant.process(openai, threadId, ingredientJson);

Handler factory

export function getHandler(type, ctx) {
  const map = {
    text: TextHandler,
    photo: PhotoHandler,
    voice: AudioHandler
  };
  return new map[type](ctx);
}

6. Research takeaways

1. Specialists beat generalists – smaller prompts + focused tools yield faster, cheaper, more accurate results than a single giant prompt.
2. Context matters – OpenAI threads let each assistant build on prior outputs without leaking user data into prompts.
3. Modalities converge – with images and audio now first‑class citizens in GPT‑4o, users expect friction‑less logging regardless of input type.

7. Roadmap

• Parallel inference – run label/dish + validation assistants concurrently to shave another second off latency.
• Personalised goals – adapt macro targets based on user‑set objectives and health metrics.
• Recipe RAG – query a private recipe KB to auto‑log complex dishes.
• Wearable integrations – merge CGM/heart‑rate data for holistic nutrition insights.

8. Try it & tell me what you think

Ready to ditch clunky food diaries? Fire up Telegram and DM @self_nutri_bot. The first 15 days are on me, and every message hits my inbox – so bug reports, feature ideas, or spicy feedback are more than welcome.

Happy logging & healthy eating! 🚀