Human Brains Can't Optimize What Machines Can See
Why the best excipient choice isn't always the safest. Navigating the impossible trade-off between cost, bioavailability, stability, manufacturability, and regulatory precedent when human cognition can only hold four variables at once.
Executive Summary
Formulation scientists balance at least five competing dimensions at once: cost efficiency, bioavailability enhancement, long-term stability, manufacturing scalability, and regulatory precedent. Human working memory caps out around four variables. The math does not work.
AI-driven multi-objective optimization can map the full trade-off space and surface formulations that sequential human reasoning would miss entirely.
A formulation that performs well in a two-milliliter vial can stall in a 1,000-liter reactor, adding years and millions of dollars to development timelines. The "best" choice in one dimension may be catastrophic in another, and this briefing explains why.
The Cognitive Constraint
"Human working memory was estimated to be limited to 7 plus or minus 2 variables in the 1950s. A more current estimate is 4 plus or minus 1 constructs. Decision quality generally becomes degraded once this limit of four constructs is exceeded."
Source: Cognitive Limitations in Clinical Decision-Making, PMC (2018)
The Nature of Multi-Objective Optimization
In pharmaceutical formulation, multiple competing factors shift across development stages and require continuous adjustment. Traditional Design of Experiments (DoE) frameworks were not built for this level of complexity.
In multi-objective optimization problems, objectives are frequently conflicting: the improvement of one objective leads to the degradation of another. They are also non-commensurable, dealing with objectives that have different units or scales of measurement. In such cases, there is usually no single optimal solution but rather a set of trade-off solutions representing a compromise between conflicting objectives. These solutions are called non-dominated solutions, forming the Pareto optimal set.
The Pareto front represents the set of all Pareto efficient solutions. When there are many distinct factors to consider in an optimization problem, a Pareto front represents the set of solutions that are "equally good" overall, albeit by making different concessions and compromises. In pharmaceutical formulation, this is particularly relevant because no single "best" solution exists; trade-offs must be made explicitly, and Pareto-optimal solutions define the boundary of what is achievable.
The Five Optimization Dimensions
Virtual screening operates in a high-dimensional search space where binding affinity, solubility, toxicity, and pharmacokinetic properties interact in ways that remain poorly characterized. Each dimension below carries its own constraints, and these constraints frequently conflict with one another.
Dimension 1: Cost Optimization
The global pharmaceutical excipients market was valued at over $9.5 billion in 2022 and is projected to grow to nearly $15 billion by 2033. The API represents 50-52% of total manufacturing cost, with excipients, labor (10-15%), and GMP compliance (10-20%) comprising the remainder.
Generic drugs, which frequently involve strategic switching of excipients, can be 80% to 85% cheaper than branded counterparts, primarily attributed to careful selection of more affordable excipients. However, many pharmaceutical companies initially develop drugs in capsule form during early clinical trials due to cost-effectiveness, then transition to tablet formulations upon commercialization. This transition, while economically beneficial, introduces reformulation risks.
Dimension 2: Bioavailability Optimization
More than 70% of new chemical entity (NCE) candidates have poor solubility and consequently poor bioavailability, making this the leading cause of Phase 1 First-in-Human trial failures. BCS Class IV drugs compound the problem: both poor solubility and poor membrane permeability, with limited formulation options to address either.
There is no single technology with universal applicability to every API displaying bioavailability issues. Techniques such as solid dispersions, micronization, complexation, and lipid-based systems each carry their own trade-offs in terms of cost, complexity, and stability.
Dimension 3: Stability Optimization
Stability encompasses five main facets: chemical, physical, microbiological, therapeutic, and toxicological. A solid is more stable in its crystalline form, and typically under the right conditions an amorphous solid will revert to its crystalline state. This creates a fundamental tension: amorphous forms offer superior bioavailability but inferior stability.
Amorphous solid dispersions (ASDs) are kinetically stabilized systems prone to phase separation, often driven by the presence of water. Phase separation during manufacturing, storage, or dissolution can result in crystallization and altered drug release, leading to compromised product performance.
Dimension 4: Manufacturability Optimization
Lab-scale success does not predict commercial-scale success. Equipment, mixing dynamics, and heat transfer all change during scale-up, and a formulation tuned for bench conditions may behave differently in pilot and production equipment.
Any major change to drug product formulation or process may require rework: repeat bio or clinical studies and data generation to support the robustness of the process and stability in filings and submissions. This creates a tension between optimizing the formulation for performance versus optimizing for ease of manufacture.
Dimension 5: Regulatory Precedent Optimization
A critical regulatory distinction that many formulators overlook: GRAS (Generally Recognized As Safe) status does NOT guarantee excipient safety in drugs. GRAS is currently used only for food additives and carries little to no regulatory weight in the drug arena.
The dose, frequency of administration, duration of administration, and route of administration often differ between food use and proposed drug product use. If an excipient is not listed in the FDA's Inactive Ingredients Database (IID), then a full nonclinical qualification program may be needed, following FDA guidance for safety evaluation of pharmaceutical excipients.
NCEs with Poor Solubility
Excipients Market (2022)
QbD Development Time Savings
Pharma AI Priority (2025)
The Human Limitation: Cognitive Constraints on Decision Making
Human short-term or working memory was estimated to be limited to 7 plus or minus 2 variables in the 1950s. A more current estimate is 4 plus or minus 1 constructs. Decision quality generally becomes degraded once this limit of four constructs is exceeded. Because of this limitation, most decisions are based on one to three variables.
This allows for rational decision rules, but it caps the number of dimensions a person can optimize simultaneously, well below the five that formulation science demands. Herbert Simon's concept of bounded rationality describes how human decision-making is constrained by cognitive abilities, information availability, and time constraints.
Satisficing, the tendency to find a "good enough" solution rather than an optimal one, is characterized by sequential rather than simultaneous action on goals. Humans have a tendency to set aspiration levels for each of the multiple goals that they face, and a tendency to operate on goals sequentially rather than simultaneously because of the bottleneck of short-term memory. While formulation science demands simultaneous optimization of five dimensions, human cognition processes them sequentially, inherently limiting our ability to find truly Pareto-optimal solutions.
Cognitive Biases in Pharmaceutical Development
Confirmation Bias
Teams generate one path forward and seek only data to support it, disregarding alternatives. Each decision builds over a thousand smaller previous decisions.
Saliency Bias
A previous success casts a rosy glow on a new, similar project, leading to over-optimism about similar approaches.
Champion Bias
A project champion's enthusiasm can override objective assessment of a formulation's weaknesses.
Comfort Bias
A formulator may demonstrate bias for a particular process because of previous experience, trendy technology, or equipment availability.
Case Study: The Ritonavir Polymorphism Disaster
Abbott Laboratories launched Ritonavir (Norvir) in 1996 as a semisolid capsule for HIV treatment. Through the entire development program and NDA filing, only one crystalline form (Form I) had ever been observed. Polymorph screening found nothing else.
In 1998, researchers found that several batches of ritonavir were failing quality control tests. The drug substance was not dissolving properly, and a solid was precipitating out of the semisolid capsules. Investigation revealed the existence of a new crystalline form (polymorph) denoted Form II. This newly discovered form was more thermodynamically stable than Form I, less than 50% as soluble, and impossible to convert back once formed.
The original formulation had been optimized for bioavailability and manufacturability. Stability received less attention. Form I dissolved better and had superior bioavailability; Form II was thermodynamically more stable. In highly supersaturated solutions, heterogeneous nucleation allowed Form II to appear spontaneously and take over.
Abbott withdrew ritonavir capsules from the market, disrupting HIV treatment for thousands of patients. Reformulating the drug and regenerating Form I required substantial time and cost. The case remains one of the most referenced examples in pharmaceutical science of what happens when a formulation is optimized along some dimensions but not all of them.
The AI Solution: Machine Learning for Pareto-Optimal Formulation Design
High-throughput automation paired with AI tests thousands of formulations in the time classical DoE covers dozens. Active learning algorithms direct each successive batch toward the most informative regions of the design space, and multi-objective optimization evaluates performance and manufacturability in the same loop.
Bayesian optimization cuts experiments from roughly 25 (typical DoE) to around 10. In one published study, researchers optimized thermal stability of three tandem single-chain Fv variants within 25 experiments, less than one-third the number required by classical DoE methods.
AI Performance Metrics
- Tm improved by 3.2 degrees C (from 59.7 to 62.9 degrees C)
- Monomer loss reduced by 15.5% (from 18.1% to 2.5%)
- Only 33 experiments required, approximately 3x smaller than traditional DoE
Human cognition processes dimensions sequentially. AI evaluates them in parallel across the full design space and learns from historical data as it goes. With three excipients, five possible concentrations, and five process parameters at three settings each, the total number of possible unique formulations reaches 3,645,000. Most real programs use more excipients and additional process parameters, pushing possibilities into the tens or hundreds of millions.
The Non-dominated Sorting Genetic Algorithm II (NSGA-II) has become a standard approach for multi-objective optimization. It combines parent and offspring populations, selects the best solutions using non-dominated rank and crowding distance, and introduces diversity in the objective space. Its value is that it generates a full Pareto front rather than a single "optimal" solution, giving formulators visibility into the complete set of trade-offs.
Industry Transformation
"By 2025, 75% of pharmaceutical companies have made generative AI a strategic priority. Since 2021, FDA has received more than 100 submissions for drug and biologic applications using AI/ML components, spanning drug development from molecule to medicine."
Quality by Design: The Integration Framework
Quality by Design replaces trial-and-error formulation with a structured, risk-based methodology. Its elements are the Quality Target Product Profile (QTPP) for identifying critical quality attributes, Product Design for critical material attributes, Process Design for critical process parameters, Control Strategy for specifications at each manufacturing step, and Process Capability for continual improvement.
Studies indicate that QbD can reduce development time by up to 40% and reduce batch failures and material wastage by up to 50%. The design space is defined by critical process parameters and their acceptable ranges. Since a design space "assures quality" of the drug product, the limits defined provide the basis for validation acceptance criteria.
The next step is integrating these tools: AI and high-throughput automation to screen thousands of formulations, active learning to direct experimentation, multi-objective optimization to evaluate against full target product profiles, and patient-centric design objectives such as ease of administration, meal-time flexibility, and tolerability.
Conclusions
Without computational support, a formulation scientist is trying to optimize across five competing dimensions with a working memory that holds 4 plus or minus 1 variables. A formulation that looks optimal across 2-3 dimensions may be far from optimal when all five are considered. Sequential cognitive processing misses interactions between dimensions, and experience-based heuristics (confirmation bias, comfort bias, saliency bias) compound the problem.
AI multi-objective optimization addresses this directly: simultaneous evaluation of all dimensions, identification of the full Pareto frontier, discovery of non-obvious solutions, 3x or greater reduction in experimental requirements, and cost savings measured in tens of millions of dollars.
AI maps the full trade-off space; human expertise decides where to operate within it. The formulation scientist's dilemma was never that any single excipient choice is dangerous. It is that the best choice in isolation is rarely the best choice in context, and seeing that context across five dimensions simultaneously requires computation that exceeds human cognitive capacity.