Measurable Outcomes of GenAI: From Hype to Business Value

In today’s episode of the “Business Tech Talks powered by BlueSoft” podcast, we discuss key insights on the practical implementation of artificial intelligence (AI) in organizations, with a special focus on the collaboration between BlueSoft and Rezon Bio, a company in the pharmaceutical sector. Below you will find a summary of the episode transcript.

Cooperation Between Rezon Bio and BlueSoft

The collaboration began about a year ago, when Rezon Bio (formerly Polpharma Bio) approached BlueSoft with a concrete proposal to build three tools supporting business processes using generative AI.

What triggered the cooperation:

• Rezon Bio had previously implemented an internal solution called Marta Chat (named after the person who came up with the idea), built entirely with internal resources.
• Marta Chat performed exceptionally well, generating over 3,000 chats per month, proving the usefulness of AI.
• As AI technology matured quickly and needs increased, the company required improvements and scaling for Marta Chat — which led them to BlueSoft.
• The project—subject to additional compliance requirements, supported by Michał Szafranko from Rezon Bio—started in February and reached production deployment in July.
• The new BlueSoft-built solution now handles up to 1,000 chats per day at peak times, confirming successful scaling.

AI Hype vs. Real Capabilities 

The speakers agree that AI is currently surrounded by enormous hype; nearly every company wants to deploy AI “immediately,” and client expectations often exceed what simple solutions can realistically deliver.

Key challenges in managing expectations:

• The biggest difficulty was managing the hype among executives, who were heavily targeted by companies trying to enter the AI market.
• Many proposed solutions turned out to be unfeasible once vendors saw how company data was actually organized (sometimes on paper).
• Educating leadership teams on data and AI models proved essential to prevent misguided investments.
• Implementing AI—especially in highly regulated pharmaceutical environments—requires strict data protection and an infrastructure ensuring sensitive data is not used to train public models.

Initiating AI Projects 

AI initiatives emerge both from executive-level pressure (influenced by media hype) and from IT departments, which monitor the market and identify needs.

Example: The Translator

• The IT department identified an issue when DeepL began sending the company emails informing how many confidential documents (their size and gigabytes) employees had uploaded into the free version.
• Instead of paying for expensive DeepL licenses or promoting inconvenient Microsoft alternatives, IT proposed and built an in-house tool that replaced DeepL and ensured data protection.
• Rezon Bio’s philosophy: AI should increase employee efficiency and democratize access to advanced tools — not reduce headcount. New tools should streamline work and enable employees to handle more tasks in the same amount of time.

Criteria for Selecting or Rejecting AI Projects

BlueSoft and Rezon Bio use agile methods, applying feedback loops and proofs of concept (POCs).

Decision Principles:

1. Low Hanging Fruit: Start with simple, short POCs that can quickly validate assumptions, gather user insights, and deliver value. (BlueSoft’s deployment in Rezon Bio was the second step after Marta Chat validated the concept.)
2. Focus on the Business Problem: Every tool must solve a real business problem — not implement AI for the sake of AI.
3. Value and ROI: Each project requires clearly defined assumptions and ROI calculations, verified with the finance department.
4. Selection: Not every project is implemented. For example, an AI-driven predictive maintenance initiative was rejected because ROI calculations showed a 12-year payback period.

Implementing AI in Regulated Environments 

In the pharmaceutical sector, the primary principle is patient safety and product quality.

Key Requirements:

• Intended Use: Regulations depend on how the tool is intended to be used.
• Human in the Loop: AI can only act as a supportive tool; human operators must always make the final decision (e.g., whether a drug batch is acceptable).
• Regulations: Supporting tools must comply with the AI Act and the upcoming pharmaceutical Annex 22.
• Data Security: Sensitive data must be strictly protected so unauthorized individuals—including executives—cannot access each other’s inputs. Due to AI’s nondeterministic nature, 100% confidence in its output is impossible, which reinforces the need for human oversight.

Path to Production Deployment 

The project followed an agile approach with two-week sprints and continuous feedback collection.

• Feedback: Several business users were involved in testing, providing comments (e.g., via Microsoft Forms), and their improvements were implemented.
• Value Delivery: The priority was achieving business goals and delivering value, not introducing “AI gadgets.”
• Collaboration: The process required trust and mutual learning—BlueSoft developed functionalities iteratively while Rezon Bio refined requirements.

Tools Developed Through the Collaboration 

Three main tools were created:

1. AI Translator:
   A secure environment (sandbox) for translating confidential documents and texts. It preserves formatting (e.g., in PowerPoint), saving significant time.
2. AI Assistant (Secure ChatGPT):
   A democratized, secure, cloud-based GPT chat. It allows attaching files and documents to conversations. A key feature is prompt templates that employees can create and share, building an internal best-practice library.
3. Knowledge Base (RAG):
   Built on several thousand documents. A critical step was cleaning and structuring data (“garbage in, garbage out”) so AI learns from high-quality sources. The solution includes custom knowledge-area tagging, enabling users to choose context (e.g., logistics) for more accurate responses — supporting an AI-first approach.

Measuring ROI of AI Investments 

ROI was measured using two complementary approaches:

1. Analytical Approach (Numbers):
   Based on market studies of productivity gains from GenAI.
   Each chat interaction was estimated to save 8 minutes of employee time, equating to roughly 10 PLN in savings.
   With 1,000 daily uses, this results in approx. 10,000 PLN saved per day.
2. Qualitative Approach (Case Studies):
   In-depth interviews gathered tangible examples of time savings — such as training materials created in 2 hours instead of 2 days.
3. Added Value:
   AI implementation also boosts competitiveness and strengthens organizational quality, though these aspects are harder to quantify.

Key Takeaways and Advice for CIOs/COOs

Guidance for leaders considering AI adoption aligns with the principle:
“Start small and think big.”

Core Advice:

• Verifiability: Begin with small, measurable solutions that deliver immediate results.
• Value Over Technology: Choose partners focused on delivering business value—not technology for its own sake.
• Business Understanding: Solutions must address real business problems to gain organizational support.
• Continuous Improvement: Collecting feedback and implementing it quickly is crucial for building effective AI products.

Employee Concerns About AI

Employees sometimes fear job loss, especially after news of layoffs in major tech companies.

How Rezon Bio addressed these concerns:

• Education: Explain that AI increases the amount of work employees can complete—sometimes 2–3 times more in the same time frame.
• Tool, Not Replacement: AI is a tool that simplifies and accelerates work, just as computers replaced typewriters.
• Adaptation: Employees must remain open to new technologies, as the ability to work with AI will become essential in a world where AI is ubiquitous.