Sunday, August 10, 2025

Creating, Analyzing and Testing Modern APIs without and with LLMs

Introduction to APIs in Microservice Architectures


In modern software development, microservice architectures have become increasingly popular due to their scalability, maintainability, and deployment advantages. At the heart of any microservice system are the APIs that enable communication between services. These APIs serve as the contract between different components, allowing them to interact while remaining independently deployable.


A microservice-oriented system typically consists of multiple small, focused services that each handle a specific business function. These services communicate with each other through well-defined APIs, most commonly RESTful APIs, though GraphQL, gRPC, and message-based APIs are also widely used. The quality of these APIs directly impacts the overall system's reliability, performance, and developer experience.


The lifecycle of an API in a microservice system involves several key phases: creation, testing, and evaluation. Each phase presents unique challenges and opportunities for optimization. In recent years, Large Language Models (LLMs) have emerged as powerful tools that can assist in all these phases, complementing traditional approaches with AI-enhanced capabilities.


This article explores both standard techniques and LLM-assisted methods for creating, testing, and evaluating APIs in microservice architectures. We'll provide detailed explanations and practical code examples to help software engineers implement these approaches in their own systems.


Creating APIs for Microservices


Standard Approaches to API Creation


The traditional approach to creating APIs for microservices typically follows an API-first design methodology. This involves defining the API contract before implementing the actual service logic. This approach ensures that the API is well-designed, consistent, and meets the needs of its consumers.


One common standard for defining RESTful APIs is the OpenAPI Specification (formerly known as Swagger). OpenAPI allows developers to define APIs in a language-agnostic way using YAML or JSON. Here's an example of a simple OpenAPI definition for a product service:



openapi: 3.0.0

info:

  title: Product Service API

  version: 1.0.0

  description: API for managing products in an e-commerce system

paths:

  /products:

    get:

      summary: Get all products

      responses:

        '200':

          description: A list of products

          content:

            application/json:

              schema:

                type: array

                items:

                  $ref: '#/components/schemas/Product'

    post:

      summary: Create a new product

      requestBody:

        required: true

        content:

          application/json:

            schema:

              $ref: '#/components/schemas/ProductInput'

      responses:

        '201':

          description: Product created successfully

          content:

            application/json:

              schema:

                $ref: '#/components/schemas/Product'

components:

  schemas:

    Product:

      type: object

      properties:

        id:

          type: string

          format: uuid

        name:

          type: string

        price:

          type: number

          format: float

        description:

          type: string

    ProductInput:

      type: object

      required:

        - name

        - price

      properties:

        name:

          type: string

        price:

          type: number

          format: float

        description:

          type: string



Once the API is defined, developers can implement the service using frameworks like Spring Boot for Java, Express for Node.js, or Flask for Python. Here's an example of implementing the above API using Spring Boot: 



@RestController

@RequestMapping("/products")

public class ProductController {

    

    private final ProductService productService;

    

    public ProductController(ProductService productService) {

        this.productService = productService;

    }

    

    @GetMapping

    public ResponseEntity<List<Product>> getAllProducts() {

        List<Product> products = productService.getAllProducts();

        return ResponseEntity.ok(products);

    }

    

    @PostMapping

    public ResponseEntity<Product> createProduct(@RequestBody ProductInput productInput) {

        Product createdProduct = productService.createProduct(productInput);

        return ResponseEntity

            .status(HttpStatus.CREATED)

            .body(createdProduct);

    }

}



This standard approach works well but requires careful planning and manual implementation of the API contract.


LLM-Assisted API Creation


Large Language Models (LLMs) can significantly streamline the API creation process. They can help generate API specifications, implementation code, and even assist in refining the API design based on best practices.


According to research by Chauhan et al. (2025), LLMs can automate the API-first development of RESTful microservices by generating OpenAPI specifications and implementing server code based on natural language descriptions [https://arxiv.org/abs/2502.09766].


Here's how an LLM might assist in creating an API:


1. A developer provides a natural language description of the desired API

2. The LLM generates an OpenAPI specification

3. The LLM then generates implementation code for the chosen framework

4. The developer reviews and refines the generated code


For example, a developer might provide this prompt to an LLM:


"Create an API for a user service that allows creating, retrieving, updating, and deleting users. Each user has an ID, name, email, and age."


The LLM could then generate both the OpenAPI specification and implementation code. Here's an example of what the LLM might generate for a Spring Boot implementation:



// User.java

public class User {

    private String id;

    private String name;

    private String email;

    private int age;

    

    // Constructors, getters, and setters

}


// UserController.java

@RestController

@RequestMapping("/users")

public class UserController {

    

    private final UserService userService;

    

    public UserController(UserService userService) {

        this.userService = userService;

    }

    

    @GetMapping

    public ResponseEntity<List<User>> getAllUsers() {

        return ResponseEntity.ok(userService.getAllUsers());

    }

    

    @GetMapping("/{id}")

    public ResponseEntity<User> getUserById(@PathVariable String id) {

        return userService.getUserById(id)

            .map(ResponseEntity::ok)

            .orElse(ResponseEntity.notFound().build());

    }

    

    @PostMapping

    public ResponseEntity<User> createUser(@RequestBody User user) {

        User createdUser = userService.createUser(user);

        return ResponseEntity

            .status(HttpStatus.CREATED)

            .body(createdUser);

    }

    

    @PutMapping("/{id}")

    public ResponseEntity<User> updateUser(

            @PathVariable String id, 

            @RequestBody User user) {

        return ResponseEntity.ok(userService.updateUser(id, user));

    }

    

    @DeleteMapping("/{id}")

    public ResponseEntity<Void> deleteUser(@PathVariable String id) {

        userService.deleteUser(id);

        return ResponseEntity.noContent().build();

    }

}



The LLM can also generate the service layer, repository, and other necessary components. This approach significantly reduces the time and effort required to create APIs, allowing developers to focus on business logic and edge cases.


Testing APIs in Microservices


Testing APIs in a microservice architecture presents unique challenges due to the distributed nature of the system. Various testing strategies are employed to ensure the reliability and correctness of APIs.


Traditional Testing Strategies


According to Martin Fowler's microservice testing pyramid [https://martinfowler.com/articles/microservice-testing/], there are several levels of testing for microservices:


1. Unit Testing

2. Integration Testing

3. Component Testing

4. Contract Testing

5. End-to-End Testing


Let's explore each of these with code examples:


Unit Testing


Unit tests focus on testing individual methods or functions in isolation. For API endpoints, this often involves mocking dependencies like services or repositories.


Here's an example of unit testing a controller using JUnit and Mockito:



@ExtendWith(MockitoExtension.class)

public class ProductControllerTest {

    

    @Mock

    private ProductService productService;

    

    @InjectMocks

    private ProductController productController;

    

    @Test

    public void testGetAllProducts() {

        // Arrange

        List<Product> expectedProducts = Arrays.asList(

            new Product("1", "Product 1", 10.0, "Description 1"),

            new Product("2", "Product 2", 20.0, "Description 2")

        );

        when(productService.getAllProducts()).thenReturn(expectedProducts);

        

        // Act

        ResponseEntity<List<Product>> response = productController.getAllProducts();

        

        // Assert

        assertEquals(HttpStatus.OK, response.getStatusCode());

        assertEquals(expectedProducts, response.getBody());

        verify(productService).getAllProducts();

    }

    

    @Test

    public void testCreateProduct() {

        // Arrange

        ProductInput input = new ProductInput("New Product", 15.0, "New Description");

        Product createdProduct = new Product("3", "New Product", 15.0, "New Description");

        when(productService.createProduct(input)).thenReturn(createdProduct);

        

        // Act

        ResponseEntity<Product> response = productController.createProduct(input);

        

        // Assert

        assertEquals(HttpStatus.CREATED, response.getStatusCode());

        assertEquals(createdProduct, response.getBody());

        verify(productService).createProduct(input);

    }

}



Integration Testing


Integration tests verify that different components of a service work together correctly. For APIs, this often involves testing the controller with the actual service implementation but mocking external dependencies.



@SpringBootTest

@AutoConfigureMockMvc

public class ProductApiIntegrationTest {

    

    @Autowired

    private MockMvc mockMvc;

    

    @MockBean

    private ProductRepository productRepository;

    

    @Test

    public void testGetAllProducts() throws Exception {

        // Arrange

        List<Product> products = Arrays.asList(

            new Product("1", "Product 1", 10.0, "Description 1"),

            new Product("2", "Product 2", 20.0, "Description 2")

        );

        when(productRepository.findAll()).thenReturn(products);

        

        // Act & Assert

        mockMvc.perform(get("/products")

                .contentType(MediaType.APPLICATION_JSON))

                .andExpect(status().isOk())

                .andExpect(jsonPath("$", hasSize(2)))

                .andExpect(jsonPath("$[0].id", is("1")))

                .andExpect(jsonPath("$[0].name", is("Product 1")))

                .andExpect(jsonPath("$[1].id", is("2")))

                .andExpect(jsonPath("$[1].name", is("Product 2")));

    }

    

    @Test

    public void testCreateProduct() throws Exception {

        // Arrange

        ProductInput input = new ProductInput("New Product", 15.0, "New Description");

        Product createdProduct = new Product("3", "New Product", 15.0, "New Description");

        

        when(productRepository.save(any(Product.class))).thenReturn(createdProduct);

        

        // Act & Assert

        mockMvc.perform(post("/products")

                .contentType(MediaType.APPLICATION_JSON)

                .content(new ObjectMapper().writeValueAsString(input)))

                .andExpect(status().isCreated())

                .andExpect(jsonPath("$.id", is("3")))

                .andExpect(jsonPath("$.name", is("New Product")))

                .andExpect(jsonPath("$.price", is(15.0)));

    }

}



Component Testing


Component testing involves testing a microservice in isolation. This means testing the service as a whole, including its API endpoints, but mocking any external services it depends on.



@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)

public class ProductServiceComponentTest {

    

    @Autowired

    private TestRestTemplate restTemplate;

    

    @MockBean

    private ExternalInventoryService inventoryService;

    

    @Test

    public void testGetProductWithInventory() {

        // Arrange

        when(inventoryService.getStockLevel("1")).thenReturn(100);

        

        // Act

        ResponseEntity<ProductWithInventory> response = restTemplate.getForEntity(

            "/products/1/with-inventory", ProductWithInventory.class);

        

        // Assert

        assertEquals(HttpStatus.OK, response.getStatusCode());

        ProductWithInventory product = response.getBody();

        assertNotNull(product);

        assertEquals("1", product.getId());

        assertEquals(100, product.getStockLevel());

    }

}



Contract Testing


Contract testing ensures that the API contract between a consumer and provider is maintained. Tools like Pact or Spring Cloud Contract are commonly used for this purpose.


Here's an example using Spring Cloud Contract with Groovy:



// Contract definition

Contract.make {

    description "should return a product by id"

    request {

        method GET()

        url "/products/1"

    }

    response {

        status 200

        headers {

            contentType applicationJson()

        }

        body([

            id: "1",

            name: "Product 1",

            price: 10.0,

            description: "Description 1"

        ])

    }

}



The contract test verifies that the provider (the product service) fulfills this contract:



@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.MOCK)

@AutoConfigureMessageVerifier

public class ProductContractTest {

    

    @Autowired

    private MockMvc mockMvc;

    

    @MockBean

    private ProductService productService;

    

    @BeforeEach

    public void setup() {

        Product product = new Product("1", "Product 1", 10.0, "Description 1");

        when(productService.getProductById("1")).thenReturn(Optional.of(product));

        

        RestAssuredMockMvc.mockMvc(mockMvc);

    }

}



End-to-End Testing


End-to-end tests verify that the entire system works correctly from a user's perspective. These tests interact with the system through its external interfaces, such as APIs, and verify that the expected results are returned.



@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)

public class OrderFlowEndToEndTest {

    

    @Autowired

    private TestRestTemplate restTemplate;

    

    @Test

    public void testCreateOrderFlow() {

        // Create a product

        ProductInput productInput = new ProductInput("Test Product", 25.0, "Test Description");

        ResponseEntity<Product> productResponse = restTemplate.postForEntity(

            "/products", productInput, Product.class);

        assertEquals(HttpStatus.CREATED, productResponse.getStatusCode());

        Product product = productResponse.getBody();

        

        // Create an order for the product

        OrderInput orderInput = new OrderInput(product.getId(), 2);

        ResponseEntity<Order> orderResponse = restTemplate.postForEntity(

            "/orders", orderInput, Order.class);

        assertEquals(HttpStatus.CREATED, orderResponse.getStatusCode());

        Order order = orderResponse.getBody();

        

        // Verify order details

        assertEquals(product.getId(), order.getProductId());

        assertEquals(2, order.getQuantity());

        assertEquals(50.0, order.getTotalPrice());

    }

}



LLM-Enhanced Testing Methods


LLMs can enhance API testing in several ways, from generating test cases to analyzing test results and identifying potential issues.


According to research on using LLMs for mining and testing constraints in API testing [https://dl.acm.org/doi/10.1145/3691620.3695341], LLMs can help identify and test constraints that might not be explicitly documented in the API specification.


Here are some ways LLMs can enhance API testing:


Generating Test Cases


LLMs can generate comprehensive test cases based on the API specification or even just a description of the API. This can help ensure better test coverage and identify edge cases that human testers might miss.


For example, given an OpenAPI specification, an LLM could generate test cases like this:



@Test

public void testCreateProductWithValidInput() {

    // Test creating a product with valid input

    ProductInput input = new ProductInput("Valid Product", 15.0, "Valid Description");

    ResponseEntity<Product> response = productController.createProduct(input);

    assertEquals(HttpStatus.CREATED, response.getStatusCode());

    assertNotNull(response.getBody());

    assertEquals("Valid Product", response.getBody().getName());

}


@Test

public void testCreateProductWithMissingRequiredField() {

    // Test creating a product with missing required field (name)

    ProductInput input = new ProductInput(null, 15.0, "Valid Description");

    ResponseEntity<Product> response = productController.createProduct(input);

    assertEquals(HttpStatus.BAD_REQUEST, response.getStatusCode());

}


@Test

public void testCreateProductWithNegativePrice() {

    // Test creating a product with negative price

    ProductInput input = new ProductInput("Valid Product", -15.0, "Valid Description");

    ResponseEntity<Product> response = productController.createProduct(input);

    assertEquals(HttpStatus.BAD_REQUEST, response.getStatusCode());

}



Automated Test Analysis


LLMs can analyze test results and provide insights into potential issues or improvements. For example, an LLM could analyze test failure logs and suggest possible causes or solutions.



Test Failure Analysis:

- Test 'testCreateProductWithNegativePrice' failed with status code 201 instead of expected 400

- Possible cause: Missing validation for negative prices in the controller or service layer

- Suggested fix: Add validation to ensure price is positive before creating the product



Conversational Testing


LLMs can enable conversational testing, where testers describe scenarios in natural language, and the LLM generates and executes the appropriate tests.


For example, a tester might say:


"Test what happens when a user tries to create a product with a very long name, like 1000 characters."


The LLM could then generate and execute a test like:



@Test

public void testCreateProductWithVeryLongName() {

    // Generate a string of 1000 characters

    String longName = "a".repeat(1000);

    

    ProductInput input = new ProductInput(longName, 15.0, "Valid Description");

    ResponseEntity<Product> response = productController.createProduct(input);

    

    // Check if the API properly handles this case

    // (either by rejecting it with a 400 or by truncating the name)

    if (response.getStatusCode() == HttpStatus.BAD_REQUEST) {

        // API rejected the long name, which is one valid approach

    } else if (response.getStatusCode() == HttpStatus.CREATED) {

        // API accepted the request, but we should check if the name was truncated

        assertNotNull(response.getBody());

        assertTrue(response.getBody().getName().length() <= 255); // Assuming 255 is the max length

    } else {

        fail("Unexpected status code: " + response.getStatusCode());

    }

}



Evaluating API Performance and Quality


Evaluating APIs is crucial for ensuring they meet performance requirements and maintain high quality. This section covers both standard evaluation techniques and LLM-based approaches.


Standard Metrics and Tools


Several standard metrics and tools are used to evaluate API performance and quality:


Performance Metrics


1. Response Time: The time it takes for an API to respond to a request.

2. Throughput: The number of requests an API can handle per unit of time.

3. Error Rate: The percentage of requests that result in errors.

4. Latency: The time delay between a request and response.


Here's an example of using JMeter to test API performance using XML:



<jmeterTestPlan version="1.2" properties="5.0">

  <hashTree>

    <TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Product API Performance Test">

      <elementProp name="TestPlan.user_defined_variables" elementType="Arguments">

        <collectionProp name="Arguments.arguments"/>

      </elementProp>

    </TestPlan>

    <hashTree>

      <ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="Users">

        <elementProp name="ThreadGroup.main_controller" elementType="LoopController">

          <boolProp name="LoopController.continue_forever">false</boolProp>

          <intProp name="LoopController.loops">10</intProp>

        </elementProp>

        <stringProp name="ThreadGroup.num_threads">100</stringProp>

        <stringProp name="ThreadGroup.ramp_time">10</stringProp>

      </ThreadGroup>

      <hashTree>

        <HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="Get All Products">

          <stringProp name="HTTPSampler.domain">localhost</stringProp>

          <stringProp name="HTTPSampler.port">8080</stringProp>

          <stringProp name="HTTPSampler.path">/products</stringProp>

          <stringProp name="HTTPSampler.method">GET</stringProp>

        </HTTPSamplerProxy>

        <hashTree/>

        <ResultCollector guiclass="SummaryReport" testclass="ResultCollector" testname="Summary Report"/>

        <hashTree/>

      </hashTree>

    </hashTree>

  </hashTree>

</jmeterTestPlan>



Quality Metrics


1. API Design Quality: Adherence to REST principles, consistency, and usability.

2. Documentation Quality: Completeness, accuracy, and clarity of API documentation.

3. Security: Protection against common vulnerabilities like injection attacks, broken authentication, etc.

4. Reliability: Consistency of API behavior under various conditions.


Tools like SonarQube can be used to analyze code quality, while security scanners like OWASP ZAP can identify security vulnerabilities.


LLM-Based Evaluation Techniques


LLMs offer new approaches to evaluating APIs, particularly for aspects that are difficult to quantify with traditional metrics.


API Design Evaluation


LLMs can analyze API designs and provide feedback on adherence to best practices, consistency, and potential issues. For example, an LLM could evaluate an OpenAPI specification and provide feedback like:



API Design Analysis:

- Endpoint naming is inconsistent: '/products' uses plural, but '/user/{id}' uses singular

- Some endpoints are missing proper error responses (400, 404, etc.)

- Authentication scheme is not consistently applied across endpoints

- Recommended improvements:

  - Standardize on plural resource names ('/users/{id}' instead of '/user/{id}')

  - Add error responses to all endpoints

  - Apply consistent authentication requirements



Documentation Quality Assessment


LLMs can evaluate API documentation for completeness, clarity, and accuracy. They can identify missing information, ambiguous descriptions, and inconsistencies.



Documentation Analysis:

- The 'price' field is described as "the product price" which is too vague

- The 'status' field is mentioned in the documentation but not defined in the schema

- The authentication process is not clearly explained

- Recommended improvements:

  - Clarify that 'price' is in USD and specify format (e.g., two decimal places)

  - Add 'status' field to the schema or remove from documentation

  - Add step-by-step authentication instructions with examples



Automated API Testing with LLMs


LLMs can generate and execute comprehensive test suites that evaluate both functional correctness and edge cases. According to research on automating REST API Postman test cases using LLMs [https://arxiv.org/abs/2404.10678], LLMs can enhance the creation of test cases by automating the generation of varied and intricate test scenarios.


Here's an example of how an LLM might generate a Postman test script:



pm.test("Status code is 200", function () {

    pm.response.to.have.status(200);

});


pm.test("Response time is acceptable", function () {

    pm.expect(pm.response.responseTime).to.be.below(500);

});


pm.test("Response has the correct structure", function () {

    const responseJson = pm.response.json();

    pm.expect(responseJson).to.be.an('array');

    if (responseJson.length > 0) {

        pm.expect(responseJson[0]).to.have.property('id');

        pm.expect(responseJson[0]).to.have.property('name');

        pm.expect(responseJson[0]).to.have.property('price');

    }

});


pm.test("Price values are positive numbers", function () {

    const responseJson = pm.response.json();

    for (let i = 0; i < responseJson.length; i++) {

        pm.expect(responseJson[i].price).to.be.a('number');

        pm.expect(responseJson[i].price).to.be.above(0);

    }

});



Conclusion and Future Directions


Creating, testing, and evaluating APIs in microservice architectures involves a combination of established best practices and emerging technologies. Traditional approaches provide a solid foundation, while LLMs offer new capabilities that can enhance and streamline these processes.


The integration of LLMs into API development workflows is still evolving, but the potential benefits are significant. LLMs can help generate code, create test cases, evaluate API designs, and provide insights that might be missed by human developers. However, they should be used as tools to augment human expertise, not replace it.


As microservice architectures continue to evolve, we can expect to see further integration of AI technologies into all aspects of API development. Future directions might include:


1. More sophisticated LLM-based code generation that considers system-wide architecture and constraints

2. Automated API evolution and versioning based on changing requirements

3. Intelligent monitoring and self-healing systems that use LLMs to detect and address issues

4. Enhanced security analysis using LLMs to identify potential vulnerabilities


By combining the strengths of traditional software engineering practices with the capabilities of LLMs, developers can create more robust, maintainable, and effective microservice systems.

No comments: