Variable</th>	Purpose</th></tr></thead>
`MONGODB_URI</code></td>`	Use a real database instead of in-memory seed data</td></tr>
`AZURE_OPENAI_API_KEY</code>, AZURE_OPENAI_ENDPOINT</code>, AZURE_OPENAI_DEPLOYMENT</code></td>`	Enable the AI chat assistant (OpenAI-compatible)</td></tr>
`VITE_SENTRY_DSN</code></td>`	Enable error and performance monitoring</td></tr>
`AUTH_HEADER_NAME</code></td>`	HTTP header for the JWT (default: Authorization</code>)</td></tr> </tbody></table> Each capability layers on independently. You can clone and replace the Task domain with your own, use the generated Cursor/Windsurf skill for AI-assisted setup, or adopt layers incrementally; the repo README has the details.</p> Extending the Template</h2> Adding a new domain entity is a six-step process:</p> Zod schema</strong> in schemas.ts</code> with .describe()</code> annotations</li> Repository methods</strong> in the interface, then implement in seed and MongoDB</li> Server functions</strong> wrapping the repository (GET for loaders, POST with invalidateMiddleware</code> for mutations)</li> AI tools</strong> that call the server functions (wrapped with withErrorHandling()</code>), plus a getCurrentUserContext</strong> tool so the AI knows who is logged in and what they can do</li> Routes</strong> for the UI pages</li> Tests</strong> for the seed repository, new utilities, and E2E flows (Playwright runs against seed data)</li> </ol> Because every layer follows the same pattern, adding a new entity takes minutes, not hours. The database, AI provider, and observability layer are behind interfaces; to swap one, implement the interface and update the factory (e.g. getRepository.ts</code> for the repository, the AI adapter factory, or the observability factory in the repo).</p> Conclusion</h2> The goal is not a framework — it's a starting point</strong>. Fork the template, replace the Task domain with your own, and you have a production-ready full-stack application with:</p> Type-safe server functions</li> JWT authentication via middleware (mutations use requireAuthMiddleware)</li> AI-powered data querying and task create/edit/delete with permission-aware errors (401/403)</li> Dark/light mode UI</li> Swappable database, AI provider, and observability</li> E2E tests via Playwright against seed data (no database needed)</li> Docker-ready deployment</li> </ul> All with zero configuration for local development.</p> The template is open source at github.com/carlosvin/tanstack-fullstack-ai-template</a>. You can try the live demo</a> to see it in action before cloning. We hope it saves you the same weeks of scaffolding it saved us.</p> Built with TanStack Start, Mantine, TanStack AI, MongoDB, Zod, Sentry, Vitest, Playwright, and Biome.</em></p> Isolated Integration Tests with pytest-scenarios 2026-01-03T00:00:00+00:00 Integration tests are essential for catching bugs that unit tests miss—but they come with a notorious problem: shared state</strong>. When tests share a database, they become flaky, order-dependent, and painful to debug.</p> What if each test could start with a clean slate, with exactly the data it needs, without manual cleanup or complex fixtures?</p> That's what pytest-scenarios</code></a> does.</p> The Problem</h2> Consider testing an e-commerce checkout system. You need to verify that:</p> Active customers can complete orders</li> Suspended customers are rejected</li> Out-of-stock items fail gracefully</li> </ul> Each scenario requires different data. With traditional approaches, you either share fixtures (tests interfere), manually clean up (boilerplate hell), or use transaction rollbacks (complex and misses commit-time bugs).</p> The Solution: Declarative Test Scenarios</h2> With pytest-scenarios</code>, you declare what state you need</em>, and the plugin handles the rest:</p> def test_checkout_rejects_suspended_customer(scenario_builder, db): # Arrange: Declare the scenario scenario_builder.create({ "customers": [{"customer_id": "user_1", "status": "suspended"}], "products": [{"product_id": "item_1", "in_stock": True}], "orders": [{"id": "order_1", "customer_id": "user_1", ...}], }) # Act: Run your business logic checkout = Checkout(db) result = checkout.process(order_id="order_1") # Assert: Verify the outcome assert result.success is False assert "not active" in result.error </code></pre> No manual cleanup. No fixture dependencies. Each test is completely isolated.</p> Key Concepts</h2> Templates Define Defaults</h3> Define templates with sensible defaults—tests only override what matters:</p> # tests/templates/customers.py TEMPLATE = { "name": "John Doe", "email": "john.doe@example.test", "status": "active", } </code></pre> Automatic Cleanup</h3> The plugin clears all managed collections before each test</em>. Even if a test fails, the next test starts fresh.</p> Real Database Testing</h3> Tests run against a real MongoDB instance, catching issues that mocks would miss: query errors, index problems, schema mismatches.</p> Try It Yourself</h2> I've created a complete runnable example that demonstrates pytest-scenarios</code> with a simple e-commerce checkout system.</p> 👉 View the example on GitHub</a></strong></p> The example includes:</p> A Checkout</code> class</a> with real business logic</li> Integration tests</a> covering happy paths and edge cases</li> Templates</a> for customers, products, and orders</li> </ul> Quick Start</h3> git clone https://github.com/carlosvin/pytest-scenarios.git cd pytest-scenarios/examples </code></pre> Then follow the instructions in the example README</a> to run the tests.</p> Installation</h2> pip install pytest-scenarios # or uv add pytest-scenarios </code></pre> Configure in pyproject.toml</code>:</p> [tool.pytest.ini_options] db-url = "mongodb://127.0.0.1:27017" db-name = "my_test_db" templates-path = "tests/templates" </code></pre> Learn More</h2> 📦 PyPI Package</a></li> 📁 GitHub Repository</a></li> 📖 Example Project & README</a></li> </ul> Stop fighting flaky integration tests. Let pytest-scenarios</code> handle the isolation while you focus on what matters: testing your business logic.</p> About 2025-08-15T00:00:00+00:00 About this blog</h2> Welcome to 'My Software Engineering Journey' blog</a>, a personal blog where I document my adventures in the world of code. Here, I dive deep into topics I'm passionate about, especially within software engineering, AI, programming languages, code examples, coding practices, recent learnings and more. From deep dives into performance and efficiency to practical guides on building and deploying applications, my goal is to share what I've learned along the way. Whether you're a seasoned developer or just starting out, I hope you'll find something valuable and inspiring in these posts</a>. Join me as I explore the ever-evolving landscape of software engineering, one line of code at a time.</p> Note that there are some old posts, that are kind of legacy, I keep them for reference, one example can be Choosing a Modern C++ stack</a>, it was written in 2017, updated on 2020, but I guess today is not modern anymore 😄.</small></p> About me</h2> Before</h3> Over the years, I've had the privilege of working with industry in great companies like Microsoft</a>, eBay</a>, Champ, Santander Global Tech and Spanish Lotteries, where I've led teams, architected solutions, and developed scalable systems that power critical business functions. My expertise spans across languages and frameworks such as Go</a>, Java</a>, React</a>, Python</a>, TypeScript</a>, C++</a> and more. I've contributed to projects ranging from e-commerce platforms to real-time trading systems, always with a focus on improving efficiency, reliability, and user experience.</p> Now</h3> Currently, I lead the Sales Apps team at MongoDB</a>. Our mission is to boost the productivity of the GTM organization by building innovative software solutions. We leverage the FARM stack</a>—FastAPI</a>, React</a>, MongoDB</a>, and Next.js</a>—along with TanStack Router</a>, to create tools that help our sales teams work more efficiently and effectively.</p> We also created a sister team with the same mission of improving GTM efficiency, but focused on leveraging AI to solve problems.</p> Contact</h2> I'm always open to new opportunities and collaborations. Feel free to reach out if you'd like to connect!</p> TanStack Router: Opinionated Guidelines for Production React Apps 2025-08-15T00:00:00+00:00 This post captures a practical way to implement a React application using TanStack Router. It focuses on maintainability, predictable data flows, and ergonomics for both reading and writing code.</p> 🔗 GitHub Repository</a></li> 🚀 Live Preview</a></li> </ul> Note:</strong> The guidelines and recommendations in this guide are based on our real-world experience building a new internal application as the Sales Apps Team at MongoDB</a>. They reflect lessons learned and best practices developed throughout that process.</p> </blockquote> Chosen tech stack</h2> Typescript</h3> Faster and safer development: Our goal was to catch bugs as early as possible—ideally at compile time, together with a next level developer experience. This approach not only reduces runtime errors but also makes refactoring and onboarding significantly easier.</p> Note:</strong> In my experience, well-documented project or team guidelines have an even greater positive impact on onboarding than end-to-end typing alone.</p> </blockquote> We can see some examples of next level code assistance enabled by TypeScript in tools like Arktype</a>, TanStack Router</a>, or openapi-typescript</a>.</p> E2E type safe: Arktype + TanStack router + openapi-fetch</h3> Any input to our application will be validated and typed, there are different data inputs that can affect our application state:</p> Search params</strong>: TanStack Router offers robust, type-safe search param validation out of the box. We use Arktype</a> for runtime validation, which integrates seamlessly with TanStack Router—see this concise, powerful example</a>.</p> </li> Backend</strong>: In this case we generate a typed client from the backend OpenAPI spec</a> with openapi-fetch</code></a>. All backend interactions are handled exclusively through this generated client, ensuring type safety and consistency across the app.</p> </li> </ul> Note:</strong> For teams seeking an extra layer of runtime validation, tools like Arktype</a> can be used to validate backend responses. However, in our experience—especially as maintainers of both the backend and frontend—this has proven unnecessary. With well-maintained OpenAPI specs, semantic versioning, and strong type generation, we've yet to encounter bugs caused by a mismatch between expected and actual responses. Unless your backend is managed by a separate team or you frequently encounter contract drift, runtime validation with Arktype is likely not needed in this case.</p> </blockquote> Testing: Vitest + React testing library</h3> Here the choices are quite obvious:</p> Vitest</a> because it is the go-to testing framework if you are using Vite</a>.</li> Testing Library</a> for intuitive, user-focused component tests.</li> </ul> Project guidelines</h2> Use search params to keep page</strong> state</h3> Favor search params over component state for sharable, restorable URLs. This reduces ad‑hoc useState</code> and makes the page state linkable.</p> This guideline is one of the easiest and most impactful decisions we have made:</p> Being able for our users to share exactly the same data via URLs was really important and we get this for free.</li> If we want to change the page state, we just navigate to change search params:</li> </ul> import { createFileRoute } from '@tanstack/react-router' import { type } from 'arktype' const productSearchSchema = type({ page: 'number = 1', }) export const Route = createFileRoute('/shop/products/')({ validateSearch: productSearchSchema, loaderDeps: ({ search: { page } }) => ({ page }), loader: ({ deps: { page } }) => fetchProducts({ page }), }) function Products () { const navigate = Route.useNavigate() function handleChangePage(page: number) { // here we are just updating the query navigate({ search: (prev) => ({...prev, page})}) } } </code></pre> When the navigation is performed, the loader will be called again to fetch the products in the new page. If the route was already loaded, then it will use the cached data.</p> Page state is fully type safe, always accessible, and can be as complex as needed. For example, this URL encodes all state to list products on page 2, 100 per page, sorted by ascending price, searching "water ski": /shop/products?page=2&itemsPerPage=100&sort=asc&query=water ski&sortBy=price</code> Share this URL and everyone sees the exact same state.</p> If I want to change the state, will only have to navigate sort by date in descending order:</p> -`/shop/products?page=2&itemsPerPage=100&sort=asc&query=water ski&sortBy=price` +`/shop/products?page=2&itemsPerPage=100&sort=desc&query=water ski&sortBy=date` </code></pre> Note:</strong> Component-level state management will still be necessary for UI interactions (e.g., form inputs, toggle states, loading indicators). This guideline specifically applies to maintaining the current page state</strong>—the data and filters that define what content is displayed and should be shareable via URL.</p> </blockquote> Interacting with the backend</h3> Keep backend calls thin and predictable. Recommended approach:</p> Generate a typed client from your OpenAPI spec (e.g., with openapi-fetch</code>).</li> Wrap responses in a small helper that normalizes errors into a single AppError</code> shape your UI can consume.</li> </ul> Example shape returned by processResponse</code>:</p> type AppError = { title: string description?: string } type Processed<T> = { data?: T; error?: AppError } async function getProduct(productId: string) { // response.data: only present if 2XX response // response.error: only present if 4XX or 5XX response const response = await client.GET("/products/{product_id}", { params: { path: { product_id: productId }, }, }) // Here is where we try to convert the error into an AppError unified interface const { data, error } = processResponse({ response, action: `Fetching product ${productId}`}) return { data, error } } </code></pre> Data fetching - Loaders</h3> Leverage route loaders</a> when possible</strong> because they simplify data flows:</p> Avoid most data contexts/hooks for reads; when the route renders, data is ready to pass down</li> Built‑in caching based on declared dependencies</a>.</li> </ul> Exceptions in loaders</h4> Throw in loaders (e.g., AppError</code>) so route error components handle failures consistently. Centralize error typing so UIs can show banners/toasts with title</code> and description</code>.</p> This is just convenient, because of TanStack Error Components</a>.</p> Applying this pattern, fetching a product now throws on error for cleaner loader usage:</p> function getProduct(productId: string) { // ... same code as in the example above, the only change is below if (error) { throw error } return data } </code></pre> Mutations</h3> Mutation functions are similar to fetchers but generally should NOT throw exceptions</strong>. They're invoked from event handlers, and throwing loses error type information</strong>. Return a { data, error }</code> object instead.</p> 🚫 Ideally, we want to avoid having to inspect or discriminate error types at runtime:</p> try { throw new NotFoundError('Not Found', 404); } catch (error: unknown) { if (error instanceof NotFoundError) { console.log('The object was not found', error.objectId); } else { console.log(`Caught something else: ${error}`); } } </code></pre> ✅ Using the mutation in a route component:</p> async function handleDelete(productId: string) { const { data, error } = await deleteProduct(productId) if (error) { // The error has a title and description attributes pushToast({ variant: 'warning', ...error }) } else { pushToast({ variant: 'success', title: 'Deleted!', description: `Product ${productId}` }) // Invalidate so the loader re-fetches the page state router.invalidate() } } </code></pre> Styling</h3> Use CSS Modules for simplicity. Create Component.module.css</code> next to Component.tsx</code>. Vite handles it out of the box, no extra config needed.</p> // HomeLayout.tsx import classes from './HomeLayout.module.css' export default function HomeLayout() { return ( <div className={classes.homeContainer}> <h2 className={classes.mainTitle}>Welcome!</h2> </div> ) } </code></pre> Leverage nested routing to show nested components</h3> If a nested component isn't rendered by default (e.g., a modal), make it a nested route instead of toggling hidden UI state.</p> Benefits:</p> Clear separation of concerns; no need to render hidden components</li> Nested routes can access parent loader data</li> Lazy loading is straightforward</li> Recoverable app state via URLs (e.g., /products</code> vs /products/new</code> for opening a modal)</li> </ul> Avoid calling a loader when the parent already loaded the data</h4> In nested routes, reuse parent data:</p> const { currentUser } = useLoaderData({ from: '__root__' }) </code></pre> Leverage the default router cache</h3> The built‑in cache works well by default. After a mutation followed by navigation, you may want to router.invalidate()</code> to refetch fresh data.</p> Improve performance by setting only the needed loader deps</h4> The dependencies are also part of the caching mechanism, and only the selected search params will re-fetch data.</p> const productsIndexSchema = type({ page: 'number = 1', sort: '"newest" \| "oldest" \| "price" = "newest"', condensedView: 'boolean = false', }) // /routes/products.tsx export const Route = createFileRoute('/products')({ validateSearch: productsIndexSchema, loaderDeps: ({ search: { page, sort } }) => ({ page, sort }), loader: ({ deps: { page, sort } }) => fetchProducts({ page, sort, }), }) </code></pre> In this example, updating the state to display a more condensed view does not trigger a data refetch, since condensedView</code> is not included in the loaderDeps</code>. This means the loader will not re-run when only the view mode changes, optimizing performance by avoiding unnecessary network requests.</p> // This is not making the loader to re-fetch data, because condensedView is not defined in the loaderDeps Route.navigate({ search: ({prev}) => ({...prev, condensedView: true})}) </code></pre> Using router history</h3> Use history navigation when it reflects user intent (e.g., closing a modal):</p> Simpler than reconstructing navigate</code> calls with params and search state</li> Preserves cache/state more often; if you do need fresh data (e.g., post‑mutation), invalidate</li> </ul> Project structure</h3> src</code> assets</code>: public assets</li> components</code>: React components; each usually has its own folder with styles and tests</li> hooks</code>: custom React hooks</li> routes</code>: TanStack Router routes (route tree definitions and pages)</li> services</code> api</code>: wrappers around backend interactions; see fetch/mutation sections</li> schemas</code>: arktype</code> schema definitions</li> </ul> </li> </ul> </li> </ul> Easily creating a golang command line tool 2021-02-14T00:00:00+00:00 Golang</a> is one of the most useful technologies I've recently learned. Golang has pretty nice support for networking, command line, or logging out of the box—you don't need any dependency. But there are libraries making developers' life even easier.</p> I've already talked about creating REST service in go</a>, today I'd like to focus on creating a command line tool.</p> Flags</h2> Command line tool to read number of lines as an input integer parameter (Golang):</p> package main import ( "flag" "fmt" ) func main() { var nFlag = flag.Int("lines", 1234, "number of lines") flag.Parse() fmt.Printf("Lines %d\n", nFlag) } </code></pre> With previous simple code we have already some useful capabilities:</p> Print help menu</h3> $ cmd --help > Usage of cmd: -lines int number of lines (default 1234) </code></pre> Use default value</h3> $ cmd > Lines 1234 </code></pre> Pass a value to the command tool</h3> $ cmd --lines=2 > Lines 2 </code></pre> Pass an invalid value</h3> $ cmd --lines=asdf > invalid value "asdf" for flag -lines: parse error Usage of cmd: -lines int number of lines (default 1234) exit status 2 </code></pre> For a simple command line tool, in most cases, we will have enough with the default language support. But if we want to bring more features to the combo like reading configuration from environment variables or from files, then a library like the awesome Viper</a> will come really handy.</p> Viper</h2> Viper is a very powerful and well-documented library widely used in many projects. I will not get into the details of how to use it, because I've created an abstraction to simplify its usage, I named it Meta-Viper</a>.</p> Meta-Viper</h2> Meta-Viper</a> abstracts you from the details of reading configuration from files, environment, or flags. This extra simplicity comes with a tradeoff: we are missing some flexibility.</p> Let's see some features with an example.</p> Firstly, create a go modules project with the meta-viper dependency</h3> go mod init example.com/meta-viper # Create a go modules project go get github.com/carlosvin/meta-viper # Install the meta-viper dependency </code></pre> Now let's create a program that is configurable from files, command line params, and environment variables.</p> main.go</strong></p> package main import ( "log" "os" config "github.com/carlosvin/meta-viper" ) // Here is where you define the struct that will hold the configuration values // cfg_name is the parameter name // cfg_desc is the parameter description that will be shown in the command line help type appConfig struct { Host string `cfg_name:"host" cfg_desc:"Server host"` Port int `cfg_name:"port" cfg_desc:"Server port"` SearchAPI string `cfg_name:"apis.search" cfg_desc:"Search API endpoint"` } func main() { // Instantiate the structure with default values cfg := &appConfig{ Host: "localhost", Port: 6000, SearchAPI: "google", } // Meta-Viper instance is loading the configuration from wherever is available: files, env, or input params _, err := config.New(cfg, os.Args) if err != nil { panic(err) } log.Printf("Loaded Configuration %v...", cfg) } </code></pre> Now let's see some examples of how this command line tool is able to load configuration.</p> Print usage (help)</h3> $ cmd --help Usage of flagsConfig: --apis.search string Search API endpoint (default "google") --config string Configuration name --config-dirs strings Configuration directories search paths (default [.,config,configs,cfg]) --host string Server host (default "localhost") --port int Server port (default 6000) pflag: help requested exit status 2 </code></pre> Run with default values</h3> $ cmd 2021/02/15 23:12:48 No configuration name has been specified, so no configuration file will be loaded. Using flags and environment variables. 2021/02/15 23:12:48 Loaded Configuration &{localhost 6000 google}... </code></pre> Read the port from environment variable and host from input param</h3> $ PORT=9999 cmd --host=myhost 2021/02/15 23:15:47 No configuration name has been specified, so no configuration file will be loaded. Using flags and environment variables. 2021/02/15 23:15:47 Loaded Configuration &{myhost 9999 google}... </code></pre> The last example will aggregate all the possible sources of configuration, it will extend the previous one adding configuration from a file. So let's create a configuration file:</p> the-config.json</strong></p> { "apis": { "search": "duckduckgo" } } </code></pre> Read configuration from the-config.json file, from environment and from input params</h3> $ PORT=9999 cmd --host=myhost --config=the-config 2021/02/15 23:22:17 Loaded Configuration &{myhost 9999 duckduckgo}... </code></pre> TIP: Here you can find a multi-environment example</a> a little bit more complete.</p> Parameterized Tests in Cypress 2021-01-06T00:00:00+00:00 Cypress</a> is a testing framework for anything running on a web browser. I am using it to test this site and I've talked before about it in the post "This is Sapper"</a>.</p> In this post I won't explain how to use Cypress</a>, they have a pretty nice documentation: Getting started with Cypress</a>.</p> I will stick to the parameterized tests topic.</p> Parameterized tests</strong>: Also known as dynamic tests, it is an useful technique where you can specify an input dataset and the test case will be repeated for each element in the dataset.</p> Explicit input data set</h2> Since sometimes you can’t run the test for all the possible inputs, you can select a set of meaningful data to test.</p> Example checking that the relevant pages in the site have valid HTML syntax:</p> describe('validate pages', () => { [ // <1> "/", "/categories", "/categories/sapper", "/posts/creating-custom-maven-plugin-default", "/posts/choosing-modern-cpp-stack", "/posts/cpp-mutex", "/posts/cpp-pragma-pack", "/posts/creating-custom-maven-plugin", "/posts/delete-html-tags-py-django", "/posts/gradle-cpp", "/posts/java-embedded-db-performance-comparison", "/posts/react-typescript-parcel", "/posts/pwa-lessons-learned-cra", ].forEach((url) => { // <2> it(`should be valid HTML ${url}`, () => { // <3> cy.visit(url) cy.htmlvalidate() }) }) }) </code></pre> <1> List with relevant pages to validate <2> Iterate over the list of pages <3> Test case execution for selected page</p> Test results</h3> </p> Generated input data set</h2> Instead of explicitly defining a data set, you can generate it, for example, creating a random data set.</p> Example checking that any generated random number is less than 0:</p> describe('random', () => { const inputData = Array.from(Array(100)).map(x => Math.random()) // <1> inputData.forEach(x => { // <2> it(`should be valid ${x}`, () => { // <3> expect(x).to.be.lessThan(1) }) }) }) </code></pre> <1> It generates an array of 100 random numbers <2> Iterate over the generated array <3> Test case execution</p> Test results</h3> </p> Tip:</strong> Please check the cypress official examples to learn more about dynamic/parameterized testing</a>.</p> </blockquote> Golang over Java for REST services 2020-11-01T00:00:00+00:00 Lately I've been working with two different technology stacks almost in parallel, in both cases we were using them to develop REST</a> services.</p> During this time I've come up with some conclusions and opinions I'd like to share.</p> A disclaimer, few months ago, I had several years of experience with Java and 0 days of professional experience with Golang.</p> Actual project examples</h2> Few months ago I created an API to extract and structure COVID-19 data</a> from ECDC website</a>. I developed it in Spring Boot (REST)</a>.</p> Few months later I had the luck of work on my first professional project in Go</a> and I decided to create a port of the API to extract COVID-19 data</a> in Go</a>, just for learning and for fun.</p> Now we have two REST services, almost functionally identical, but developed in two different tech stacks, so we can easily compare some relevant aspects of both.</p> Source code for the 2 REST services implementations</h3> Java</a> + Spring Boot (REST)</a></th> Go</a> + Gin framework</a></th></tr></thead> https://github.com/carlosvin/covid-rest</a></td> https://github.com/carlosvin/covid-rest-go</a></td></tr> </tbody></table> Note</strong>: I actually created that COVID-19 data REST API</a> to be the data source for the COVID19-Stats App</a>, a PWA</a> built with Svelte</a>, but that's another topic.</p> </blockquote> The Ecosystems</h2> If you want to create a REST</a> service just in plain Java you will have extra work to do, in Golang a little bit less. That's why we use framework, because they've already solved many common problems for us.</p> For this comparison I am going to use Spring Boot (REST)</a> for Java</a> and Gin framework</a> for Go</a>, but in both languages there are a lot of production ready nice options.</p> Routing</h2> Go - Without framework</h3> Go uses the concept of HTTP</a> multiplexer or router. You can specify routes using patterns and link those routes to handlers. The router will decide which handler has to execute the request based on the path received.</p> router.go file</h4> package main import ( "log" "net/http" ) func main() { router := http.NewServeMux() router.Handle("/redirect", http.RedirectHandler("https://carlosvin.github.io/", 307)) router.HandleFunc("/hello", func(w http.ResponseWriter, r http.Request) { w.Write([]byte("Hello world!")) }) log.Println("Listening...") http.ListenAndServe(":3000", router) } </code></pre> Source code is already quite simple, but there might more complex routing use cases.</p> Go - Gin Framework</h3> Happily there are frameworks that help us to keep our base code simple, for example when we need to extract path parameters, which is quite common use case in REST</a> API</a>s, we can use a routing library, I've used Gorilla Mux</a> and Gin framework</a> and I liked more Gin framework</a>.</p> Full example</a></h4> import ( "github.com/carlosvin/covid-rest-go/handlers" "github.com/carlosvin/covid-rest-go/readers" "github.com/gin-gonic/gin" ) func main() { // ... r := gin.Default() r.GET("/countries", router.Countries) r.GET("/countries/:code", router.Country) r.GET("/countries/:code/dates", router.CountryDates) r.GET("/countries/:code/dates/:date", router.CountryDate) r.Run() } </code></pre> And this is a handler example</a>, the router.Countries one</h4> func (r routerImpl) Countries(cgin.Context) { c.JSON(200, r.countries()) } </code></pre> Java + Spring.io</h3> The Spring Boot (REST)</a> is based on the concept of Controller, it is implemented using annotations on the class and methods.</p> Controller Example</h4> @Validated @RestController // (1) @RequestMapping("/countries") // (2) public class CountriesController { // Some source code is not shown, you can find the complete example in the repository @GetMapping("/{country}/dates/{isoDateStr}") // (3) public DateStatsDto getDateByCountry(@Size(min = 2, max = 2) @PathVariable String country, @Size(min = 10, max = 20) @PathVariable String isoDateStr) throws NotFoundException { return new DateStatsDto(service.getDate(country, DateUtils.convert(isoDateStr))); } } </code></pre> Declare the class as Controller so it is registered in Spring Boot (REST)</a></li> Controller base path definition</li> Handler definition for a nested path under the main controller path. Spring Boot (REST)</a> makes easy to extract path variables defined in the route, you can directly use them as method arguments.</li> </ol> Validations</h2> Go - Gin Framework Validation</h3> Gin framework</a> uses an external validation package validator</a>, besides that it is fully integrated with Gin framework</a>.</p> type User struct { Name string `validate:"required"` // (1) Email string `validate:"required,email"` } err := validate.Struct(user) // (2) validationErrors := err.(validator.ValidationErrors) // (3) </code></pre> The validation system uses Go</a> tags, it is not the same as Java</a> annotations, but in the validation case, it works in pretty same way as annotations.</li> Executes the validation explicitly</li> Extracts validation errors</li> </ol> Java + Spring.io Validation</h3> You can enable the validation in the controller level, then in the handlers you can also specify the type of validation. Let's explain it using the previous example:</p> CountriesController.java Validation Example</h4> @Validated // (1) @RestController @RequestMapping("/countries") public class CountriesController { // Some source code is not shown, you can find the complete example in the repository @GetMapping("/{country}/dates/{isoDateStr}") public DateStatsDto getDateByCountry( @Size(min = 2, max = 2) @PathVariable String country, // (2) @Size(min = 10, max = 20) @PathVariable String isoDateStr) throws NotFoundException { return new DateStatsDto(service.getDate(country, DateUtils.convert(isoDateStr))); } } </code></pre> Declare the class as Controller so it is registered in Spring Boot (REST)</a></li> @Size</code> validates that the input argument country has 2 characters</li> </ol> The validation system is more powerful than you can see in this code snippet, for example adding @Valid</code> annotation opens the door to complex types validation.</p> Filtering and Middleware</h3> Different approaches, pretty much the same end result.</p> I will elaborate this topic in following days.</p> Dependency injection / IoC</h2> Spring IoC</a></h3> Spring IoC</a> is the most complete and powerful systems I've ever used for IoC</a>, actually, the first time I used Spring professionally was just to deal with IoC</a>. It supports XML configuration files or Java</a> annotations, I like annotations more, here a simple example from Spring IoC</a> documentation:</p> Spring IoC example</h4> @Repository public class JpaMovieFinder implements MovieFinder { // (1) // implementation elided for clarity } // public class SimpleMovieLister { private MovieFinder movieFinder; @Autowired // (2) public void setMovieFinder(MovieFinder movieFinder) { this.movieFinder = movieFinder; } // ... } </code></pre> JpaMovieFinder</code> is instantiated by Spring IoC</a></li> With @Autowired</code> annotation Spring IoC</a> knows that has to inject movieFinder</code> argument. It should be a class implementing MovieFinder</code></li> </ol> Go</h3> Neither Go</a> nor Gin framework</a> has any IoC</a> solution, but you can still apply Dependency Injection</a> technique to decouple your components and improve the testability of your system.</p> Dependency injection simple example in Go</a></h4> package main import "fmt" // Greeter interface to greet the caller type Greeter interface { greet() } type greeterHello struct{} func (g greeterHello) greet() { // (3) fmt.Println("Hello!") } type greeterHi struct{} func (g greeterHi) greet() { // (4) fmt.Println("Hi!") } // App Application representation type App struct { greeters []Greeter // (1) } func (app App) startup() { for _, v := range app.greeters { v.greet() } } func main() { greeters := []Greeter{ // (2) &greeterHello{}, &greeterHi{}, &greeterHello{}} app := &App{greeters} app.startup() } </code></pre> App</code> accepts an array of Greeter</code></li> During App</code> instantiation we pass different implementations of Greeter</code></li> Greeter implementation that prints Hello!</strong></li> Greeter implementation that prints Hi!</strong></li> </ol> It is more verbose, but there is an advantage, there is nothing hidden, everything is explicit and you have full control of instantiation order.</p> As soon as you use Dependency Injection</a>, I don't have any strong opinion about using IoC</a> system or doing Dependency Injection</a> manually.</p> Testing</h2> Unit tests</h3> For unit tests there are no big differences.</p> Go</a> comes with a standard library for testing and benchmarking</a>.</p> For Java</a> there are many well-known unit testing frameworks, but Spring already has quite big support for unit testing</a>.</p> Integration tests</h3> Go Integration Testing</h4> There are no support for Integration Tests in Go</a>, you will have to implement everything by yourself, although it is not difficult, here you can find a simple example</a>.</p> Spring</h4> On the other hand, Spring has a great testing support</a>.</p> To write integration tests for REST</a> services, MockMvc</a> is really convenient.</p> I've used MockMvc in the covid-rest project</a>.</p> MockMvc code snippet from CountriesControllerTest.java</a></h4> @Autowired private MockMvc mockMvc; // (1) @Test void getCountries() throws Exception { this.mockMvc.perform(get("/countries")) // (2) .andDo(print()).andExpect(status().isOk()) // (3) .andExpect(jsonPath("$.", hasSize(144))) .andExpect(jsonPath("$.ES.confirmedCases",comparesEqualTo(9191))) .andExpect(jsonPath("$.ES.deathsNumber", comparesEqualTo(309))) .andExpect(jsonPath("$.ES.countryCode", comparesEqualTo("ES"))) .andExpect(jsonPath("$.ES.countryName", comparesEqualTo("Spain"))) .andExpect(jsonPath("$.ES.path", comparesEqualTo("/countries/ES"))) .andExpect(jsonPath("$.VC.countryName", comparesEqualTo("Saint Vincent and the Grenadines"))) .andDo(document("countries/list", preprocessResponse(prettyPrint(), new CropPreprocessor()))); } </code></pre> The Spring test runner injects the MockMvc object.</li> We use MockMvc to call to the endpoint we have created.</li> Then we validate the endpoint response: status code and body.</li> </ol> Performance</h2> Besides the languages specific differences, the main difference is the performance. The CPU consumption in Go</a> is smaller, but about the memory the difference is really significant, the order of 30 times smaller fingerprint</strong>.</p> Memory</h3> Here I've found a surprising difference, just by checking the memory consumption in my laptop.</p> Gin framework</a> 15.6MB</code></li> Spring Boot (REST)</a> 465.9MB</code></li> </ul> </p> Speed</h3> Following the TechEmpower benchmarks</a>:</p> Gin framework</a> is in 193 position, 9.9%.</li> Spring Boot (REST)</a> is in 284 position, 4%.</li> </ul> Following the The Benchmarker results</a>:</p> Gin framework</a>: position 33.</li> Spring Boot (REST)</a>: position 68.</li> </ul> Conclusions</h2> If I were you, I'd choose Go</a> if:</p> If you value the explicit over implicit, keep in mind that there is a cost, you will most likely have to write more lines of code.</li> If you value the simplicity, Go</a> has a quite reduced set of keywords, which reduces the learning curve and simplifies the code reviews.</li> If RAM memory usage is critical for your project, actually I'd just keep away from Spring Boot (REST)</a>.</li> If the project you are going to work on is a distributed system, specially if it is based on HTTP</a>.</li> </ul> Create a PWA with Create React App: Lessons Learned 2019-11-04T00:00:00+00:00 I started working with React</a> few year ago, always the project creation was from scratch, not using any template/scaffolding. Regarding bundlers normally I used either Webpack</a> at work or Parcel</a> for personal projects.</p> Few months ago, I wanted to start a personal project to keep track of my travel expenses. I was in a kind of rush because at that time, I was almost in the middle of my gap year, I wanted to focus on implementing main functionality and get an MVP</a> (minimum viable product) the sooner the better, so I thought it was the right time to try out Create React App</a> or CRA</a>.</p> Note</strong>: CRA</a> allows you to have a production ready PWA</a> in React</a> quickly, which is awesome. They take care of configuration and package dependencies, you only have to take care of dependencies you need for your project and of course, implement your project, CRA</a> is good, but is not magic.</p> </blockquote> As I said, I wanted to be implementing business logic ASAP, so together with using CRA</a>, I also took other decisions/shortcuts driven by the need of speeding up the development pace, I will talk about those choices in following sections describing some drawbacks and benefits.</p> Chosen Technology Stack for Budget Tracker</a></h2> CRA</a></li> React</a></li> Typescript</a></li> Material UI</a></li> Victory</a></del> Frappe charts</a></li> Firestore</a></li> Firebase Authentication</a></li> </ul> So far I am quite happy with the outcome, but with the lessons learned while developing this app, in the future, with enough time, most likely I will not choose same technology stack again</strong>. You can try the application Budget Tracker</a> and judge for yourself.</p> Along this post I will describe what are, in my experience, the benefits and drawbacks of taking these shortcuts and technical decisions.</p> Create React Application: CRA</a></h2> Create React App</a> doesn’t support Web Workers</a> neither allows to customize Service Worker</a> implementation without ejecting</a>.</p> Service worker</h3> You might want to customize your Service Worker</a> to send/receive post messages</a>, to perform background sync</a> or show web notifications</a>. In that case, you will have to eject your project</a> and maintain the configuration by yourself, which might imply a little bit of headache.</p> There are other options to customize service worker and avoid ejecting CRA</a>, but they are not straightforward enough for my taste.</p> Web worker</h3> If you need to perform any heavy processing without blocking the main thread, you can just use a Web Workers</a>, but this feature is not supported by CRA</a>. The Web Workers</a> can communicate with main thread using post messages</a> and it can also show web push notifications</a>.</p> There are also other options to use Web Workers in CRA and not eject</a>, but they imply quite some extra work.</p> Webpack</h3> Webpack</a> is the bundler used by CRA</a>. You don’t need to know much about it, unless you eject your project</a>, then you will have to deal with Webpack configuration file</a>, this is just a warning, just in case you are not comfortable with it.</p> Firebase</h2> Budget Tracker</a> supports data synchronization between different devices, so it requires a backend</a> side to deal with authentication and to save/read data remotely. I considered two options: Firebase</a> or implement REST</a> API.</p> For this backend</a>, I chose Firebase</a> because it is super easy to implement, because there is nothing to implement. You only have to configure authentication methods</a> and security rules</a> for Firestore</a>.</p> But Firebase</a> brings some drawbacks you must know before choosing it.</p> Drawbacks</h3> Bundle size</h4> I got really shocked first time I analyzed Budget Tracker</a> bundle size after integrating it with Firebase</a>, it grew around a 39%!</strong></p> 27% from Firestore</a> library.</li> 12% from Firebase Authentication</a> library.</li> </ul> Happily Budget Tracker</a> implementation is following code-splitting</a> principle, so user experience was not really affected with this integration. But user’s device will eventually have to download this extra 39% (539KB</strong>).</p> Offline first, not really</h4> Important</strong>: This section is not relevant if your use case doesn’t imply saving data linked to the user.</p> </blockquote> Firestore</a> requires user to be authenticated, but user can be anonymous</a>, this is really cool feature if you don’t want to force the user to identify to use the app.</p> Another very useful and cool Firestore</a> feature is that it supports offline mode</a>, so data can be saved and read even there is no Internet connection.</p> Anonymous user + offline mode features will allow an application to work as offline first.</p> So… what is this Offline first, not really</em> issue about? Let me explain a tricky scenario. First time the application is opened, Firebase</a> needs to authenticate the user, to do so, user’s device has to be connected to Internet, so you have to consider following scenario</strong> and either be OK with it or deal with it:</p> PWA</a> is installed in your device.</li> User is not authenticated.</li> User’s device is offline.</li> User opens the PWA</a> and tries to save some data.</li> That data won’t be saved correctly</strong>, because there is no user to link the data with, not even an anonymous user, because application needs to call Firebase</a> API to create an anonymous user.</li> </ol> This is not big deal, because it will seldom occur. If you want to deal with it anyway, check next section explaining how and why I did deal with this scenario.</p> How did I deal with this issue with Budget Tracker?</h5> First of all, this might not be an issue for your use case</strong>, because it will happen only first time application is loaded. I just wanted Budget Tracker</a> to be fully offline first, because it brings other benefits.</p> Implementation details</h6> Implement 2 persistence layers: Local (IndexedDB</a>) and Remote (Firestore</a>).</li> Save always data locally, regardless user authentication status.</li> If there is any authenticated user, after saving to local layer, propagate same action to remote layer (Firestore</a>) asynchronously.</li> </ul> Benefits</h6> If user is not authenticated, Budget Tracker</a> won’t load Firestore</a> client bundle. As I explained before, it is 27% of application size.</li> Application reads and writes are faster, because latest valid data is always saved locally. Clarification</strong>: Save data in Firestore</a> is also fast, because data is also cached locally, but it does a little bit more than just saving to IndexedDB</a> and you need an authenticated user.</li> </ul> </li> </ul> You can find a more detailed performance report</a>, where I analyze 3 different implementations:</p> Only Firestore</a> client.</li> Local (IndexedDB</a>) and remote (Firestore</a>) persistence layers.</li> Same as previous one, but remote layer implemented in service worker.</li> </ol> The performance results were in general better for option 2.</p> Data model</h4> Firestore API is easy and intuitive, I really like it, but don’t assume it will have same features as other document DBs or SQL DBs.</p> Check if Firestore limitations</a> fit into your data model, or if it is not too late, define your data model following Firestore best practices</a> and having those limitations in mind.</p> Firebase alternatives</h3> Besides implementing a REST</a> API, there are other services similar to Firebase</a> with smaller client bundle size and other features which might fit better to your requirements.</p> Consider other alternatives:</p> Based on Apache CouchDB</a>: PouchDB</a>, Cloudant</a>.</li> Based on Parse server</a>: back4apps</a>.</li> </ul> UI Components Library: Material UI</a></h2> I chose Material UI</a>.</p> Material UI</a>, from their web</p> React components for faster and easier web development. Build your own design system, or start with Material Design.</p> </blockquote> There were two reasons which drove me to use Material UI</a>:</p> To create simple UI components which are accessible, responsive</a> and with a consistent design is tricky and time consuming.</li> It has SVG set of Material Icons</a>. Budget Tracker</a> allows to create categories defined by a name and a selectable icon, so this icon set was really convenient.</li> </ul> There are some drawbacks</strong>, not very important in my opinion, maybe the most annoying for me is the first one:</p> Jest Snapshots</a> + Material UI</a>: The snapshots are generated with Material UI</a> CSS class names, but CSS classes order might not be deterministic, so a test might pass in your local host but not in CI</a> host. They are working on solve this issue, more info at github</a>.</li> Performance: There are some performance issues in Github</a>. During last months, whilst I’ve been using this library, I can say they are working hard on fix them and bring new features.</li> UI components libraries are complex and do quite a lot work, so most of them are quite heavy. Material UI bundle size weights: 304.2kB minified</a>. You can find some recommendations to reduce bundle size at Material UI website</a>.</li> </ul> Charts library</h2> Many of the chart libraries I’ve found are really powerful and complete, but they are also heavy because they depend on other third party libraries like D3</a>.</p> Initially I chose Victory</a>, but I realized that I only needed charts to show percentages and time series and Victory</a>’s bundle size is 468KB minified</a>.</p> After quick search in the Internet I discovered other lighter alternatives:</p> Frappe charts</a> 63KB minified</a>.</li> Chartist</a> 39KB minified</a>.</li> </ul> I tried them and I liked both. I chose Frappe charts</a> because I thought its default color scheme fits better with Budget Tracker</a> theme.</p> Both libraries come with more chart types than just bars and XY axis, take a quick look at their websites if you are interested about their supported chart set and to check how they look like.</p> Conclusion</h2> I will try to come up with a conclusion better than: "It depends"</em>, "Your use case will tell you"</em> and so on.</p> That said. It depends on your needs :p.</p> Seriously, let’s play "do not go for … if …"</em>:</p> Do not go for CRA if</h3> You want to customize Service Worker</a> for Background sync</a> or showing push notifications</a>.</li> You want to use Web Workers</a>.</li> </ul> Do not go for Firestore if</h3> You are aiming for your app to be hit by many users and you don’t know the estimated amount of reads/writes, otherwise you might get surprised with the bill. Firestore</a> scales like charm, maybe your budget doesn’t.</li> Bundle size is critical for your web application. Remember that bundle size is not that critical if you are implementing a PWA</a>, because your app files are cached.</li> </ul> Do not go blindly for the best charting library</h3> First of all, check what kind of charts you need. In many applications you are OK with XY axis chart, time series, bars or pie charts. You can easily get an smaller bundle size by just using a simple charting library like Frappe charts</a> or Chartist</a></p> Tip</strong>: Just check what are your requirements, if you are not sure about them, [Chosen Technology Stack for Budget Tracker</a>](#chosen-technology-stack-for-budget-tracker) consists of awesome products which most likely will fit your use case.</p> </blockquote> What next?</h2> My next technology stack bet goes for Svelte</a>/Sapper</a>, it is promising project, the results for small projects are really impressive, mainly in regards to bundle size, it is ridiculously small and development experience is quick and intuitive.</p> I’ve created a tiny PWA</a> to estimate currency exchange loss</a> when you go to a money changer shop: currency-loss.netlify.com</a>. Note, I got that app up and running in few hours, thanks to Svelte</a>.</p> Create SPA with React + Typescript + Parcel 2019-01-01T00:00:00+00:00 I was about to start yet another personal project, it consists of a SPA (Single Page Application) for a travel journal.</p> Some time ago I tried Parcel</a>, I really loved how simple it was to create a simple project from scratch, using Typescript</a> + React</a> stack. I’ve decided to create this template or base project, so next time I want to create a new SPA with my favorite frontend stack, I will only have to:</p> git clone https://github.com/carlosvin/react-typescript-parcel-template.git </code></pre> Read this before: Parcel is not as mature as Webpack</h2> If you want to create a production ready React application, use Webpack</a> or better create-react-app</a> which bring everything you need to develop a PWA</a> with React and Typescript. Following you can find an example of an app I am developing using create-react-app: https://github.com/carlosvin/budget-tracker</a>.</p> Parcel is a package bundler under development, not as mature as webpack. There are no go errors for me, at least in regards to Typescript support, see this issue in github #1378</a>.</p> I still think it is a promising project, bringing more simplicity and speed to JS bundlers world, I will give it a try again for serious projects when Parcel 2 is ready, check Parcel 2 development status</a>.</p> Quick start</h2> Development server</h3> git clone https://github.com/carlosvin/react-typescript-parcel-template.git cd react-typescript-parcel-template yarn install yarn start </code></pre> Last yarn start</code> command will:</p> start a development server at http://localhost:1234</code> with hot module replacement</a></li> build automatically development javascript files with source maps</li> </ul> Tip:</strong> Each time you save a file, you will see automatically the result at http://localhost:1234</code> without refreshing the page</p> </blockquote> Build production bundle</h3> yarn build </code></pre> Parcel’s default optimizations</a> will be applied to generated files.</p> Files are saved at dist</code> folder. Inside dist</code> folder there is also a file with information about bundle content sizes: dist/report.html</code>.</p> Step by step project creation</h2> In this section I will describe how I created this project.</p> Firstly, create package.json</code> with yarn init</a> command.</p> yarn init </code></pre> Add required dependencies</h3> Add React dependencies.</p> yarn add @types/react @types/react-dom react react-dom </code></pre> Previous command modifies package.json</code> file adding dependencies</code> section and will also install React packages in node_modules</code> folder.</p> { "name": "project_name", "version": "1.0.0", "main": "index.js", "license": "MIT", "dependencies": { "@types/react": "^16.7.18", "@types/react-dom": "^16.0.11", "react": "^16.7.0", "react-dom": "^16.7.0" } } </code></pre> Add Typescript compiler as development dependency.</p> yarn add --dev typescript </code></pre> We also need Parcel bundler</a>.</p> yarn add --dev parcel-bundler </code></pre> I’ve added a non-required dependency, it is a plugin to generate a report of generated bundle contents (the parcel version of webpack-bundle-analyzer</a>).</p> yarn add --dev parcel-plugin-bundle-visualiser </code></pre> Create application source code</h3> First we create the React application in src/index.tsx</code> file.</p> import * as React from "react"; import * as ReactDOM from "react-dom"; class App extends React.PureComponent { render() { return <h1>Hello world!</h1>; } } ReactDOM.render( <App />, document.getElementById("app") ); </code></pre> Parcel can take index.html</code> file as entry file and it figures out how to build the application, so let’s create src/index.html</code> as follows:</p> <html> <body> <div id="app"></div> <script src="./index.tsx"></script> </body> </html> </code></pre> We need div</code> tag for React to inject the DOM elements. The script</code> declaration is used by Parcel to find entry point to build.</p> Add commands build the project</h3> I’ve added the commands:</p> build</code>: Check "Build production bundle"</em> section.</li> start</code>: Check "Development server"</em> section.</li> </ul> "scripts": { "start": "parcel src/index.html", "build": "parcel build src/index.html" } </code></pre> Then to it is really easy to:</p> run development server: yarn start</code></li> generate a production bundle: yarn build</code></li> </ul> There is another approach described in Parcel documentation</a> that consists of installing Parcel globally.</p> I’ve opted for more isolated approach that affects only project you are working on, you just install Parcel as devDependency</code>. There is a tiny drawback, you can’t just run parcel index.html</code>, because it is not installed in your system, but in node_modules</code>.</p> There is a simple way to run any binary installed in node_modules</code>, you can just run npx parcel index.html</code>.</p> I like more to define build steps in package.json</code> file, so you can have well defined commands more suited to build your project. You can also use these commands as documentation how to build your project.</p> Configure Typescript (optional)</h3> Create a tsconfig.json</code> file:</p> { "compilerOptions": { "outDir": "./dist/", "sourceMap": true, "noImplicitAny": true, "module": "commonjs", "target": "es5", "jsx": "react" } } </code></pre> With this configuration, Typescript compiler will:</p> Generate files in dist</code> folder.</li> Generate source maps</a>.</li> Will not allow to declare any</code> type, for example following declaration is not allowed: const elements: any;</code></li> Generated module code will be CommonJs</a>.</li> Generated code will be ECMAScript</a> 5 compliant.</li> Support JSX</a> in .tsx</code> files.</li> </ul> Full source code</h2> You can find full example at: https://github.com/carlosvin/react-typescript-parcel-template</a>.</p> Or you can directly download the source code:</p> zip</a></li> tar.gz</a></li> </ul> Custom Maven Plugin - Override default build lifecycle 2018-05-12T00:00:00+00:00 I explained in previous article Example how to create custom Maven Plugin</a> which overrides site lifecycle</a>.</p> I have created another example to demonstrate how to override default Maven build lifecycle</a>. Default build lifecycle is used to construct your software project, for example, it is executed when you run mvn install</code> in a jar</code> type project.</p> You can find source code example at https://github.com/carlosvin/lifecycle-maven-plugin</a>.</p> Tip:</strong> I've also created an archetype</a> so you can easily play with the example.</p> </blockquote> Create an example project</h2> Generate the project using archetype</h3> mvn archetype:generate \ -DarchetypeGroupId=com.github.carlosvin.archetype \ -DartifactId=lifecycle-maven-plugin-archetype \ -DarchetypeVersion=0.6 </code></pre> Example how to create custom Maven Plugin 2018-03-11T00:00:00+00:00 Maven</a> has lots of plugins to assist you in project construction, testing, packaging and deployment. For example if you want to compile C++ code instead of Java, you can use native-maven-plugin</a>. But what if you need something more specific? Then you can create a custom Maven plugin.</p> I will explain how to create a simple custom maven plugin to generate static blog site from Markdown files. I know we can already do that with maven-site-plugin</a> since version 3.3, I will just use it for learning purposes.</p> You can find the source code of this example at https://github.com/carlosvin/blog-maven-plugin</a>.</p> Maven plugin concepts</h2> Mojo:</strong> An executable goal in Maven, e.g: mvn your-plugin:your-mojo</code> will execute a maven goal your-mojo</code> declared as part of your-plugin</code>.</p> Goal:</strong> It is equivalent to Mojo execution.</p> Lifecycle:</strong> It is a well-defined sequence of phases. Each phase consists of a sequence of goals. Let's see an example of lifecycle, e.g: FooLifecycle</code> has clean</code>, prepare</code> and assemble</code> phases. Each of those phases has one of more goals. FooLifecycle</em>:</p> clean: rmSources: a goal to remove source files</li> rmBuild: a goal to remove files in cache directory</li> </ul> </li> prepare: installDependencies: a goal to download dependencies for the project</li> </ul> </li> assemble: build: a goal to compile source files</li> </ul> </li> </ul> To define a custom life-cycle similar to previous one, we will use src/main/resources/META-INF/plexus/components.xml</code>, we will speak about that file in following sections. Normally is enough to override predefined lifecycles</a>, in this example, we will override site life-cycle</em>.</p> Tip:</strong> You can find an introduction to Maven life-cycles at Maven life-cycle guide</a></p> </blockquote> Create your custom plugin (Site Lifecycle)</h2> The plugin we are about to explain will override site lifecycle</a>, which has only 2 default phases, so when we run mvn site</code> using our new custom plugin it will execute the goals we are about to create.</p> Our plugin will work with md</code> (for Markdown</a>) file bindings: It will build and deploy the project using maven deployment plugin</a>.</p> Project structure</h3> src/main/java</code>: Where Java source code is</li> src/main/resources/META-INF/plexus/components.xml</code>: file to create/override maven lifecycles and artifact types. Here we can specify which goals will be executed when for an artifact type, for example, we can say that for an artifact of type whatever</code> when we run mvn foo</code> it will verify the files, run tests, run linter, compile and zip all generated files.</li> src/test/java</code>: Unit tests folder.</li> src/it</code>: Folder with all integration tests. Those integration tests are running actual projects and checking that outputs are as expected.</li> pom.xml</code>: File with Maven project description (Project Object Model</a>)</li> </ul> Dependency Injection</h3> Maven has finally chosen JSR-330</a> as dependency injection standard</a> (previously it was Plexus Annotations API).</p> To use dependency injection with Maven we have to:</p> Add javax.inject</code> dependency to pom.xml</code>, so we can use @Inject</code>, @Named</code>, and @Singleton</code> annotations in plugin implementation Java code.</p> pom.xml (Dependency)</h3> <dependency> <groupId>javax.inject</groupId> <artifactId>javax.inject</artifactId> <version>1</version> </dependency> </code></pre> Set up the sisu-maven-plugin</code> to index the JSR-330 components you want made available to Maven.</p> pom.xml (Plugin)</h3> <plugin> <groupId>org.eclipse.sisu</groupId> <artifactId>sisu-maven-plugin</artifactId> <version>0.3.3</version> <executions> <execution> <id>generate-index</id> <goals> <goal>main-index</goal> </goals> </execution> </executions> </plugin> </code></pre> Add annotations to your Mojo, e.g:</p> BuildMojo.java (Dependency Injection)</h3> @Mojo(name = "build", defaultPhase = LifecyclePhase.COMPILE) // <1> public class BuildMojo extends AbstractMojo { private final FileSetManager fileSetManager; private final MdToHtml mdToHtml; @Inject // <2> public BuildMojo(FileSetManager fileSetManager, MdToHtml mdToHtml) { this.fileSetManager = fileSetManager; this.mdToHtml = mdToHtml; // } } </code></pre> This annotation is not a dependency injection one, we will explain later what it is for.</li> It will inject an instance of FileSetManager and MdToHtml.</li> </ol> Write a custom Mojo</h3> It is quite straightforward to implement a Mojo class, we have to:</p> 1. Implement Mojo interface</h4> Your Mojo class has to implement org.apache.maven.plugin.Mojo</code>, although it is more convenient to extend org.apache.maven.plugin.AbstractMojo</code>, an abstract class to provide most of the infrastructure required to implement a Mojo except for execute method. That interface and class are described at Mojo API</a>.</p> public class BuildMojo extends AbstractMojo { // ... } </code></pre> 2. Configure Mojo with Java 5 annotations</h4> Annotate Mojo class with @Mojo</code> and input parameters with @Parameter</code>. Those annotations belong to another set of annotations to configure Mojos, Plugin Tools Java5 Annotations</a>.</p> /** * Generate HTML files from Markdown files / @Mojo(name = "build", defaultPhase = LifecyclePhase.COMPILE) // <1> public class BuildMojo extends AbstractMojo { /* * Output directory path where HTML files are generated / @Parameter( // <2> defaultValue = "${project.reporting.outputDirectory}", // <3> property = "siteOutputDirectory", // <4> required = true) private File outputDirectory; /* * A specific <code>fileSet</code> rule to select files and directories. * Fileset spec: https://maven.apache.org/shared/file-management/fileset.html / @Parameter private FileSet inputFiles; // } </code></pre> Configures Mojo name and default life-cycle phase. To execute the Mojo in this example we will use mvn site:build</code>: site</em> is the plugin name and build</em> is name</code> parameter.</li> We use @Parameter</code> annotation to pass configuration parameters to Mojo.</li> You can use properties placeholder or any String. If the parameter type is not a String, then Maven will try to cast it.</li> It allows configuration of the Mojo parameter from the command line by referencing a system property that the user sets via the -D option. E.g: mvn site:build -DsiteOutputDirectory=/var/www/html</code> will set siteOutputDirectory attribute to /var/www/html</code>.</li> </ol> Tip:</strong> More info in Maven Plugin development guide in Parameters section</a>.</p> </blockquote> 3. Implement execute</em> method</h4> As I have explained before at 1. Implement Mojo interface</strong>, our Mojo class extends org.apache.maven.plugin.AbstractMojo</code> which has one unimplemented method from org.apache.maven.plugin.Mojo</code> interface. In that method we are going to implement the Maven goal logic.</p> Mojo class instance is called from Maven execution life-cycle by invoking execute()</code> method. Before calling execute()</code>, Maven has performed some other tasks related with the Mojo:</p> Maven instantiates Mojo and injects dependencies (see Dependency Injection</strong> section).</p> Mojo mojo = new BuildMojo(fileSetManager, mdToHtml); </code></pre> Maven configures the Mojo by assigning values to parameters.</p> I will simplify execute</code> method implementation in the sample project in github</a>, because it is more complicated and this complexity is not relevant for learning purposes.</p> Maven calls execute method: mojo.execute()</code>.</p> BuildMojo.java (Execute Method)</h3> public void execute() throws MojoExecutionException { // <1> if (inputFiles == null) { setDefaultInput(); } inputDirPath = Paths.get(inputFiles.getDirectory()); String[] includedFiles = fileSetManager.getIncludedFiles(inputFiles); // <2> outputDirPath = outputDirectory.toPath(); if (includedFiles == null \|\| includedFiles.length == 0) { getLog().warn("SKIP: There are no input files. " + getInputFilesToString()); // <3> } else { if (!outputDirectory.exists()) { // <4> outputDirectory.mkdirs(); } try { for (String f : includedFiles) { convertToHtml(Paths.get(f), outputDirectory); // <5> } } catch (InterruptedException e) { throw new MojoExecutionException(e.getLocalizedMessage(), e); // <6> } } } </code></pre> If there is any error during execution, it should throw MojoExecutionException.</li> A way to get all selected files from FileSet</a>.</li> AbstractMojo supplies logger functionality.</li> If output directory doesn't exist, it will be created.</li> It converts each file Markdown to HTML.</li> Convert thrown exception to MojoExecutionException.</li> </ol> Unit tests</h2> In the example we use JUnit 4</a>, but you can use any other testing framework.</p> Firstly, you have to add the unit test library dependency to pom.xml</code>.</p> pom.xml</h3> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.11</version> <scope>test</scope> </dependency> </code></pre> Then you just have to write your unit tests under src/test/java</code> folder: src/test/java/com/maven/plugins/blog/PathsTest.java</a>.</p> To run the unit tests you just need to execute mvn test</code>.</p> Integration tests</h2> The 2 most popular ways to perform integration tests on custom maven plugins are using maven-failsafe-plugin</a> or maven-invoker-plugin</a>.</p> I've chosen maven-invoker-plugin because for me it is more straightforward. There is an answer at stackoverflow where they explain thoroughly the differences between them</a>.</p> How does Invoker Plugin work?</h3> We create projects to use our custom plugin under src/it</code> folder, so our plugin will be applied to test projects.</li> Invoker plugin will simulate a previously configured Maven execution.</li> After Maven execution, a validation script will check if our plugin outputs are the expected ones. For example, if our plugin is supposed to generate a file named foo.file</code>, verification plugin will check if that file exists, if it doesn't, integration test will fail.</li> </ol> Configure Invoker Plugin</h3> <plugin> <artifactId>maven-invoker-plugin</artifactId> <version>3.0.1</version> <configuration> <postBuildHookScript>verify</postBuildHookScript> <!-- 3 --> <showVersion>true</showVersion> <streamLogs>true</streamLogs> <noLog>false</noLog> <showErrors>true</showErrors> </configuration> <executions> <execution> <id>integration-test</id> <goals> <goal>install</goal> <!-- 1 --> <goal>run</goal> <!-- 2 --> </goals> </execution> </executions> </plugin> </code></pre> invoker:install</code> will be executed during the phase pre-integration-test and will install the main project artifact into target/local-repo.</li> invoker:run</code> will be executed during the integration-test phase and it will execute all defined integration tests under src/it</code> folder.</li> It configures invoker plugin to execute validation script after integration test project execution. This script may be written with either BeanShell or Groovy (verify.groovy or verify.bsh).</li> </ol> We have used other properties to show errors, show maven log and save it to a file.</p> You can check all invoker:run</code> configuration properties at maven-invoker-plugin run-mojo</a>.</p> Create an Integration Test Project</h3> It is a project we use to execute custom plugin goals, so we can validate if it produces the expected output.</p> There are 3 important files matching with AAA</a> phases ("Arrange-Act-Assert").</p> Arrange: pom.xml</h4> This file is a project using our custom plugin.</p> src/it/md-html/pom.xml</a></p> <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.maven.plugins.it</groupId> <artifactId>simple-it</artifactId> <version>1.0-SNAPSHOT</version> <build> <plugins> <plugin> <groupId>@project.groupId@</groupId> <artifactId>@project.artifactId@</artifactId> <version>@project.version@</version> </plugin> </plugins> </build> </project> </code></pre> It is a very simple pom file where we use placeholders to reference to our plugin under test. When invoker plugin executes following pom file, firstly will replace those placeholders to reference to the latest version of our custom plugin which was recently installed in the local repository:</p> <plugin> <groupId>com.maven.plugins</groupId> <artifactId>blog</artifactId> <version>0.0.1-SNAPSHOT</version> </plugin> </code></pre> In that way invoker plugin ensures it is testing the latest version of current project.</p> Act: invoker.properties</h4> It configures how test project will be executed.</p> src/it/md-html/invoker.properties</a></p> invoker.goals = blog:build invoker.name = Test build MD </code></pre> It will execute mvn blog:build</code>, a goal defined in our custom plugin under example or what is the same, it will execute BuildMojo described in section Write a custom Mojo</strong>.</p> Assert: verify.groovy</h4> It is the script to check that plugin execution generated the expected results.</p> Verification script, it is checking if target/site/README.html</code> file was generated by the plugin.</p> src/it/md-html/verify.groovy</a></p> File generated = new File( basedir, "target/site/README.html" ); assert generated.isFile() </code></pre> Choosing a Modern C++ stack 2017-09-15T00:00:00+00:00 I'm starting a new project in C++, but I've run into a couple of questions before starting:</p> Which build system should I use?</li> Which unit testing framework?</li> </ol> Tip</strong>: If you just want a project template so you can have a C++ project skeleton ready in seconds, just go to uuid-cpp</a> and follow the instructions in README.md</a>.</p> </blockquote> Choosing Build System (Meson</a>)</h2> I have used before Make</a>, Maven</a>, Scons</a>, Gradle</a> and Autotools</a>, but I have some reasons to try something else:</p> Autotools:</strong> It is not easy to configure and maintain. There are several configuration files and several configuration steps.</li> Gradle:</strong> CPP feature is still incubating. Not very fast. You can check a similar example project at Build C++ project with Gradle</a>.</li> Make:</strong> I don't love the syntax. Files tend to get messy as the project grows.</li> Scons:</strong> It is just slower and not as easy to understand as Meson.</li> Maven:</strong> It is slow and you might end up "Javatizing" your C++ project structure.</li> </ul> Note</strong>: I've listed just things I don't like; those projects have other great features.</p> </blockquote> CMake</a> vs Meson</a></h3> After discarding previous ones, I'm considering Meson or CMake. Both are fast build systems:</p> Although Meson is written in Python</a>, it generates a Ninja</a> build project. The first time you configure the project you have to run Meson, but for building or testing you are actually running Ninja.</p> CMake can also generate Ninja files among other formats; check CMake generators documentation for more information</a>.</p> CMake:</strong> It has a big advantage over Meson; it is mature and widely used in many projects, which means there are many examples and it will fulfill your C++ project building needs.</li> Meson:</strong> It is a young project compared with CMake, but it is growing quite fast and it has been adopted in other big projects like Gnome</a>, which has an initiative to port from Autotools to Meson</a>.</li> </ul> Finally, I've chosen</strong> Meson because syntax is really clear to me; when I read a meson.build</code> file I can quickly understand what is happening during the build process.</p> Steps to compile and test a project</h3> meson build . # First time you configure the project cd build ninja build # Each time you build it ninja test # Each time you run tests </code></pre> First time you configure the project</li> Each time you build it</li> Each time you run tests</li> </ol> Other build systems comparisons</h3> I've found two interesting comparisons about available C++ build systems; they might be a little biased because those comparisons come from Meson and Scons.</p> C++ build systems comparison from Scons</a></li> C++ build systems comparison from Meson</a></li> </ul> Unit Testing Framework</h2> I have used some xUnit</a>-based libraries like UnitTest++</a>, CppUTest</a> or Google Test</a>, which pair perfectly with Google Mock</a>. If you want a safe bet that fulfills almost all of your testing needs I highly recommend Google Test.</p> But some time ago I found a testing framework with some interesting features: Catch</a>:</p> It is just a header file with no external dependencies, so very easy to start (wget + include downloaded file).</li> You can use normal unit test style or BDD</a>-style.</li> </ul> If you want to know more about Catch, I recommend you to give it a try; it takes just 2 minutes to have a simple example up and running</a>. You can also read some interesting articles like Why do we need yet another C++ test framework?</a> or Testing C++ With A New Catch</a>.</p> Doctest</a>: A Catch alternative</h3> There is another testing framework named Doctest, with the same benefits as Catch, but it promises to be faster and lighter (performance results</a>) than Catch.</p> Doctest is modeled after Catch and some parts of the code have been taken directly, but there are differences</a>.</p> It hasn't been easy to decide; both are really similar. Below you can see some differences:</p> @@ -1,12 +1,12 @@ -#define CATCH_CONFIG_MAIN // It tells Catch to provide a main() - only do this in one cpp file +#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN -#include "catch.hpp" +#include "doctest.h" #include "Uuid.h" #include <string> constexpr int MAX_ITERS = 100; -TEST_CASE("Uuid", "[uuid]") +TEST_CASE("Uuid") @@ -26,7 +26,7 @@ TEST_CASE("Uuid", "[uuid]") // BDD style -SCENARIO("UUID creation", "[Uuid]") +SCENARIO("UUID creation") </code></pre> I've finally chosen Doctest because it promises to be faster: performance results</a>.</p> Note</strong>: I've created the project using both frameworks; you can find them in the corresponding branches: doctest branch</a> or catch branch</a>.</p> </blockquote> Example</h2> I've created an example to illustrate this article: uuid-cpp</a>. It is a basic implementation of UUID pseudo-random generator based on mt19937</a>, which is not cryptographically secure.</p> Project output artifacts</h3> When we install the project using Meson (Ninja), we will get some artifacts generated and copied in our system:</p> Shared library: libuuid</code>.</li> Header library for developers who want to use the shared library: include/Uuid.h</code>.</li> Executable uuidgen</code> (UUID</a> generator).</li> Test executable (not installed). It tests the shared library.</li> </ul> For example, if you execute:</p> /usr/local/lib/libuuid.so /usr/local/include/Uuid.h /usr/local/bin/uuidgen </code></pre> Project structure (Fork project</a>)</h3> ...existing code...</p> Filesystem in C++17 2017-05-28T00:00:00+00:00 Introduction</h2> Since C++17 new filesystem abstractions will be added to C++ environment. So far they are available as Experimental C++ Features</a>. If you want to dig more about this new library, here it is the final draft of File System Technical Specification</a>.</p> Gettting started with Experimental Filesystem Features C++17 (g++)</h2> We just have to "tell" compiler that:</p> we write C++17 (-c++1z</code>) and</li> it has to add standard library with filesystem library</em> (-lstdc++fs</code>).</li> </ul> g++ -std=c++1z main.cpp -lstdc++fs && ./a.out </code></pre> Let's see a simple example with std::filesystem::path</code> class.</p> #include <experimental/filesystem> #include <iostream> namespace fs = std::experimental::filesystem; using namespace std; int main() { fs::path aPath {"./path/to/file.txt"}; cout << "Parent path: " << aPath.parent_path() << endl; cout << "Filename: " << aPath.filename() << endl; cout << "Extension: " << aPath.extension() << endl; return 0; } </code></pre> Compile and run: Basic C++17 example</a></h3> $ g++ -std=c++1z main.cpp -lstdc++fs && ./a.out $ ./a.out Parent path: "./path/to" Filename: "file.txt" Extension: ".txt" </code></pre> C++17 Filesystem Features</h2> In this section, we are going to explain some std::filesystem</a> features with examples, which will help us to highlight differences between C++11 and C++17 so we can get a better idea about what this new library will supply and how it might make developer's work easier.</p> std::filesystem::path</h3> Upper we have seen a tiny use case for std::filesystem::path</a>. That is a quite powerful and convenient feature that supplies an multi-platform abstraction for paths to files using the correct directory path separator depending on the platform we are building our application for (\</code> for Windows based systems and /</code> for Unix based systems).</p> Directory separator</h3> When we want our application to use the correct directory separator in C++11, we could use conditional macro declaration:</p> Platform independent directory separator in C++11</h4> #include <iostream> using namespace std; #ifdef _WIN32 const string SEP = "\\"; #else const string SEP = "/"; #endif int main() { cout << "Separator in my system " << SEP << endl; return 0; } </code></pre> Compile and run: C++11 separator example</a>.</p> Platform independent directory separator in C++17. Cleaner and simpler</h4> #include <experimental/filesystem> #include <iostream> namespace fs = std::experimental::filesystem; using namespace std; int main() { cout << "Separator in my system " << fs::path::preferred_separator << endl; return 0; } </code></pre> Compile and run: C++17 separator example</a>.</p> Directory Separator Operator</h3> std::filesystem::path</a> implements /</code> operator, which allows to easily concatenate paths to files and directories.</p> When we want to concatenate paths in C++11, we have to add extra logic to avoid adding duplicate separators and to select the correct separator for target platform:</p> Concatenate paths in C++11</h4> #include <iostream> using namespace std; #ifdef _WIN32 const string SEP = "\\"; #else const string SEP = "/"; #endif int main() { string root {"/"}; string dir {"var/www/"}; string index {"index.html"}; string pathToIndex{}; pathToIndex.append(root).append(SEP).append(dir).append(SEP).append(index); cout << pathToIndex << endl; return 0; } </code></pre> Compile and run: Concatenate paths in C++11</a></h4> //var/www//index.html </code></pre> Checking program output we notice it is not fully correct, we should have checked whether path parts already contains a separator so we don't append another separator again. That logic is already implemented in std::filesystem::path</a>, so C++17 can be like:</p> Concatenate paths in C++17</h4> #include <experimental/filesystem> #include <iostream> namespace fs = std::experimental::filesystem; using namespace std; int main() { fs::path root {"/"}; fs::path dir {"var/www/"}; fs::path index {"index.html"}; fs::path pathToIndex = root / dir / index; cout << pathToIndex << endl; return 0; } </code></pre> Compile and run: Concatenate paths in C++17</a></h4> "/var/www/index.html" </code></pre> Code is cleaner and just correct, there are no duplicated separators.</p> Create/Remove Directories</h3> std::filesystem</a> comes with some utilities to create and remove files and directories, but firstly let's try to do so in C++11.</p> Create and remove nested directories in C++11</h4> #include <iostream> #include <cstdio> #include <sys/stat.h> using namespace std; int main() { auto opts = S_IRWXU \| S_IRWXG \| S_IROTH \| S_IXOTH; mkdir("sandbox", opts); mkdir("sandbox/a", opts); mkdir("sandbox/a/b", opts); mkdir("sandbox/c", opts); mkdir("sandbox/c/d", opts); system("ls -la sandbox/"); remove("sandbox/c/d"); remove("sandbox/a/b"); remove("sandbox/c"); remove("sandbox/a"); remove("sandbox"); system("ls -la"); return 0; } </code></pre> Compile and run: Create and remove directories C++11</a></h4> g++-4.9 -std=c++11 main.cpp -lm && ./a.out sandbox/a: total 12 drwxr-xr-x 3 2001 2000 4096 May 28 12:27 . drwxr-xr-x 4 2001 2000 4096 May 28 12:27 .. drwxr-xr-x 2 2001 2000 4096 May 28 12:27 b sandbox/c: total 12 drwxr-xr-x 3 2001 2000 4096 May 28 12:27 . drwxr-xr-x 4 2001 2000 4096 May 28 12:27 .. drwxr-xr-x 2 2001 2000 4096 May 28 12:27 d total 8012 drwxrwxrwx 2 2001 2000 4096 May 28 12:27 . drwxrwxrwx 3 2002 2000 8175616 May 28 12:27 .. -rwxr-xr-x 1 2001 2000 8168 May 28 12:27 a.out -rw-rw-rw- 1 2001 2000 517 May 28 12:27 main.cpp </code></pre> We have to create/remove one by one. We could rewrite this code snippet with less lines (using a loop), but we still have to pay attention to creation/deletion order, we cannot remove parent directory before we have removed all children.</p> Since C++17, we can create and remove nested directories with just one call.</p> Create and remove nested directories C++17</h4> #include <experimental/filesystem> #include <iostream> namespace fs = std::experimental::filesystem; using namespace std; int main() { fs::create_directories("sandbox/a/b"); fs::create_directories("sandbox/c/d"); system("ls -la sandbox/"); cout << "Were directories removed? " << fs::remove_all("sandbox") << endl; system("ls -la"); return 0; } </code></pre> Compile and run: Create and remove nested directories C++17</a></h4> g++ -std=c++1z -fconcepts -fgnu-tm -O2 -Wall -Wextra -pedantic -pthread -pedantic-errors main.cpp -lm -latomic -lstdc++fs && ./a.out sandbox/a: total 12 drwxr-xr-x 3 2001 2000 4096 May 28 16:45 . drwxr-xr-x 4 2001 2000 4096 May 28 16:45 .. drwxr-xr-x 2 2001 2000 4096 May 28 16:45 b sandbox/c: total 12 drwxr-xr-x 3 2001 2000 4096 May 28 16:45 . drwxr-xr-x 4 2001 2000 4096 May 28 16:45 .. drwxr-xr-x 2 2001 2000 4096 May 28 16:45 d Were directories removed? 1 total 10132 drwxrwxrwx 2 2001 2000 4096 May 28 16:45 . drwxrwxrwx 3 2002 2000 8175616 May 28 16:45 .. -rwxr-xr-x 1 2001 2000 2170976 May 28 16:45 a.out -rw-rw-rw- 1 2001 2000 393 May 28 16:45 main.cpp </code></pre> Full example: Recursive Directory Iterator</h2> This example consists of iterate recursively through dicrectories fintering files by extension.</p> C++11</h3> To keep C++11 example simple, I haven't added filtering logic, but filtering logic is present in C++17 example:</p> filesystem.11.cpp</h4> #include <dirent.h> #include <cstring> #include <iostream> #include <fstream> // std::ofstream #include <vector> #include <memory> #include <system_error> #include <sys/stat.h> using namespace std; const string UP_DIR = ".."; const string CURRENT_DIR = "."; const string SEP = "/"; string path(initializer_list<string> parts) { string pathTmp {}; string separator = ""; for (auto & part: parts) { pathTmp.append(separator).append(part); separator = SEP; } return pathTmp; } vector<string> getDirectoryFiles(const string& dir, const vector<string> & extensions) { vector<string> files; shared_ptr<DIR> directory_ptr(opendir(dir.c_str()), [](DIR dir){ dir && closedir(dir); }); if (!directory_ptr) { throw system_error(error_code(errno, system_category()), "Error opening : " + dir); } struct dirent dirent_ptr; while ((dirent_ptr = readdir(directory_ptr.get())) != nullptr) { const string fileName {dirent_ptr->d_name}; if (dirent_ptr->d_type == DT_DIR) { if (CURRENT_DIR != fileName && UP_DIR != fileName) { auto subFiles = getDirectoryFiles(path({dir, fileName}), extensions); files.insert(end(files), begin(subFiles), end(subFiles)); } } else if (dirent_ptr->d_type == DT_REG) { // here we should check also if filename has an extension in extensions vector files.push_back(path({dir, fileName})); } } return files; } int main () { auto opt = S_IRWXU \| S_IRWXG \| S_IROTH \| S_IXOTH; mkdir("sandbox", opt); mkdir("sandbox/a", opt); mkdir("sandbox/a/b", opt); vector<string> e_files = { "./sandbox/a/b/file1.rst", "./sandbox/a/b/file1.txt", "./sandbox/a/file2.RST", "./sandbox/file3.md", "./sandbox/will_be.ignored" }; // create files for (auto &f: e_files) { ofstream of(f, ofstream::out); of << "test"; } cout << "filtered files: " << endl; for (auto &f: getDirectoryFiles(".", {".rst", ".RST", ".md"})){ cout << "\t" << f << endl; } return 0; } </code></pre> Compile and run C++11 example</a></h4> g++ -std=c++11 -O2 -Wall -Wextra -pedantic -pthread -pedantic-errors main.cpp -lm -latomic -lstdc++fs && ./a.out filtered files: ./main.cpp ./sandbox/file3.md ./sandbox/will_be.ignored ./sandbox/a/b/file1.rst ./sandbox/a/b/file1.txt ./sandbox/a/file2.RST ./a.out </code></pre> C++17</h3> Following example also filters files by extension.</p> filesystem.17.cpp</h4> #include <dirent.h> #include <cstring> #include <iostream> #include <fstream> // std::ofstream #include <vector> #include <memory> #include <system_error> #include <sys/stat.h> using namespace std; const string UP_DIR = ".."; const string CURRENT_DIR = "."; const string SEP = "/"; string path(initializer_list<string> parts) { string pathTmp {}; string separator = ""; for (auto & part: parts) { pathTmp.append(separator).append(part); separator = SEP; } return pathTmp; } vector<string> getDirectoryFiles(const string& dir, const vector<string> & extensions) { vector<string> files; shared_ptr<DIR> directory_ptr(opendir(dir.c_str()), [](DIR dir){ dir && closedir(dir); }); if (!directory_ptr) { throw system_error(error_code(errno, system_category()), "Error opening : " + dir); } struct dirent dirent_ptr; while ((dirent_ptr = readdir(directory_ptr.get())) != nullptr) { const string fileName {dirent_ptr->d_name}; if (dirent_ptr->d_type == DT_DIR) { if (CURRENT_DIR != fileName && UP_DIR != fileName) { auto subFiles = getDirectoryFiles(path({dir, fileName}), extensions); files.insert(end(files), begin(subFiles), end(subFiles)); } } else if (dirent_ptr->d_type == DT_REG) { // here we should check also if filename has an extension in extensions vector files.push_back(path({dir, fileName})); } } return files; } int main () { auto opt = S_IRWXU \| S_IRWXG \| S_IROTH \| S_IXOTH; mkdir("sandbox", opt); mkdir("sandbox/a", opt); mkdir("sandbox/a/b", opt); vector<string> e_files = { "./sandbox/a/b/file1.rst", "./sandbox/a/b/file1.txt", "./sandbox/a/file2.RST", "./sandbox/file3.md", "./sandbox/will_be.ignored" }; // create files for (auto &f: e_files) { ofstream of(f, ofstream::out); of << "test"; } cout << "filtered files: " << endl; for (auto &f: getDirectoryFiles(".", {".rst", ".RST", ".md"})){ cout << "\t" << f << endl; } return 0; } </code></pre> Compile and run C++17 example</a></h4> g++ -std=c++11 -O2 -Wall -Wextra -pedantic -pthread -pedantic-errors main.cpp -lm -latomic -lstdc++fs && ./a.out filtered files: ./main.cpp ./sandbox/file3.md ./sandbox/will_be.ignored ./sandbox/a/b/file1.rst ./sandbox/a/b/file1.txt ./sandbox/a/file2.RST ./a.out </code></pre> Multi-Domain Docker Containers 2016-11-24T00:00:00+00:00 Use case</h2> We have several server applications in the same development environment, each application is bundled in a Docker container, e.g: "Container A"</strong> and "Container B"</strong>.</p> With Docker those applications have the same IP address. One way to differentiate and access to an specific application is exposing different ports.</p> </p> If we want to call to "Application A"</strong> we will do: GET http://10.20.30.40:8080/colors/red</code>.</li> If we want to call to "Application B"</strong> we will do: GET http://10.20.30.40:8081/fruits/tomato</code>.</li> </ul> But that solution is a little bit confusing, does 8080 mean we are accessing to "application A"?</p> It would be simpler and easier</strong> to remind something like:</p> Calling "Application A"</strong>: GET http://a.domain.com/colors/red</code>.</li> Calling "Application B"</strong>: GET http://b.domain.com/fruits/tomato</code>.</li> </ul> </p> Get that extra semantic value is much simpler than I thought at the beginning and you will see below.</p> How to Configure Multi-Domain Reverse Proxy</h2> I said it is easy, because we almost have to do nothing, another container will do it for us, especifically we are going to use nginx-proxy</a>, it will automatically generate the required NGINX</a> configurations.</p> So, we will have 2 applications + 1 proxy, that is 3 containers.</p> </p> Note</strong>: You can download the full example at https://github.com/carlosvin/docker-reverse-proxy-multi-domain</a>.</p> </blockquote> Example Project Structure</h3> docker-compose.yaml</strong>: Main configuration file describing architecture in previous picture.</li> a</strong>: Application A directory. Dockerfile</strong>: Container A configuration file.</li> </ul> </li> b</strong>: Application B directory. Dockerfile</strong>: Container B configuration file.</li> </ul> </li> </ul> View Project</a></p> Architecture Configuration (docker-compose)</h3> The relationships between containers is the most interesting part in this example.</p> docker-reverse-proxy-multi-domain/docker-compose.yaml</h3> a: build: a # (4) environment: VIRTUAL_HOST: a.domain.com # (1) restart: always b: build: b # (5) environment: VIRTUAL_HOST: b.domain.com # (2) restart: always nginx-proxy: # (3) image: jwilder/nginx-proxy ports: - "80:80" - "443:443" volumes: - /var/run/docker.sock:/tmp/docker.sock:ro restart: always privileged: true </code></pre> Configure the domain name for app a.</li> Configure the domain name for app b.</li> From this line there is proxy configuration (copy/paste part).</li> We tell docker-compose has to build Docker images within specified directory.</li> For example, we are saying that docker-compose has to build a Docker image using ../b/Dockerfile file.</li> </ol> Application Image Configuration</h3> a/Dockerfile</h3> FROM httpd:2.4 # (1) RUN echo "<html><body><h1>A</h1>App A works!</body></html>" > /usr/local/apache2/htdocs/index.html # (2) </code></pre> We import an image with an apache server.</li> It serves a file that prints "Host A" as default page.</li> </ol> The configuration for application B is pretty much the same:</p> b/Dockerfile</h3> FROM httpd:2.4 RUN echo "<html><body><h1>B</h1>App B works!</body></html>" > /usr/local/apache2/htdocs/index.html </code></pre> Adding domain names to your development environment configuration</h3> In Linux we just have to map the local address to domain names you have chosen, in the example a.domain.com</code> and b.domain.com</code>.</p> /etc/hosts</h3> 127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 127.0.0.1 a.domain.com # (1) 127.0.0.1 b.domain.com </code></pre> We just added last 2 lines.</li> </ol> Everything ready</h3> Now we just have to test the example:</p> docker-compose build docker-compose up </code></pre> The 3 containers are running now.</p> So we can open our favorite web browser and go to a.domain.com</code>. It will show App A works!</strong>. If we go to b.domain.com</code> then we will see App B works!</strong>.</p> </p> </p> Note</strong>: In most of the Linux distros you will need privileges to run Docker commands (sudo</code>).</p> </blockquote> Convert files formats - Windows to Unix 2016-02-12T00:00:00+00:00 If you are developing from a Windows environment to a Unix target environment, most likely you have had this issue: You install source files in Windows format in your Unix environment.</p> Windows and Unix systems use different line endings: Windows uses carriage return and line feed (\r\n</code>), while Unix uses just line feed (\n</code>). This difference can cause issues with scripts, code, and configuration files when moving files between systems.</p> There is a quite simple way to convert all your files from Windows to Unix format:</p> find . -type f -print0 \| xargs -0 dos2unix </code></pre> I got it, of course, from this Stack Overflow answer</a>.</p> REST URLs 2015-08-16T00:00:00+00:00 First time I designed a REST</a> API</a> I made several mistakes, of course. Following I'm going to explain common mistakes and what I've learned about REST</a> URL</a> with examples.</p> REST Basics</h2> Using URL</a>s for get resources.</li> Using verbs</em> for modify resources.</li> The verbs</em> are provided by the HTTP</a> protocol.</li> The verbs</em> have a direct equivalency with CRUD</a> (Create, Read, Update, Delete).</li> To access to an existent resource we need an identifier.</li> </ul> REST Verbs</h3> POST: Create</strong> new resources. GET: Read</strong> already existing resources. PUT: Update</strong> already existing resources. DELETE: Delete</strong> already existing resources.</p> It is clearer in the following table:</p> REST Verb</th> CRUD Action</th> Resource must exist</th></tr></thead> POST</td> Create</td> No</td></tr> GET</td> Read</td> Yes</td></tr> PUT</td> Update</td> Yes</td></tr> DELETE</td> Delete</td> Yes</td></tr> </tbody></table> Accessing to Resources</h3> A resource is what we want to get</em>. For example, a car.</p> To be able to get a car, that information is not enough, you can't go to your car dealer and ask for whatever car, you have to specify which one you want:</p> Good morning. I'd like to have a Fiat Bravo 1.9 Emotion 120CV.</p> </blockquote> In this manner the seller knows which one is.</p> "Fiat Bravo 1.9 Emotion 120CV" is the identifier</strong>.</p> Transferring the example to REST</a> API</a>s:</p> GET https://cardealer.com/api/cars/fiat-bravo-19-emotion-120cv </code></pre> Now our API</a> can supply the car info.</p> This is a very simple example, but actually when we access to a specific resource, we have to use something to identify it, a common and recommendable practice is use UUID</a>.</p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f </code></pre> But our API</a>, like a shop, it hasn't to be so strict. We can ask for cars with several features:</p> Good morning, I want a Fiat Bravo.</p> </blockquote> Then, the dealer kindly will show you all Fiat Bravo he has available. Let's see how API</a> says that.</p> GET https://cardealer.com/api/cars/?brand=fiat&model=bravo </code></pre> API</a> will return all cars with Fiat brand and Bravo model.</p> Brand and model are so called query parameters</strong>.</p> As you might already notice, to get resource information, we have always used GET</strong> verb</em></p> Update resources</h3> The API</a> should also support updating resources. Like reading resources, to update a resource we have to specify which resource we want to update, so we again need an identifier</em>.</p> Before, we wanted to get information (read) and we used GET</strong> verb</em>. Now the only difference is the verb.</p> We want to update</strong> so we use the equivalency HTTP</a> verb: PUT</strong>.</p> PUT https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f </code></pre> Actually something else is missing, we have to say what thing of the car we want to change, for example, let's imagine we want to change the engine power and set it to 100CV.</p> We have to send the new engine power to following URL</a> https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f</a> through HTTP</a> using PUT</strong> verb.</p> HTTP</a> protocol allows sending data within PUT message, we have to choose a sending format.</p> We can use JSON</a> or XML</a> or whatever, we only have to ensure that sent format is expected in server side.</p> Note:</strong> Designing a REST</a> API</a> requires to select a data format.</p> </blockquote> JSON example:</strong></p> { "enginePower": 100 } </code></pre> Delete Resources</h3> Let's imagine that now we are the car dealer and we don't want to sell the Fiat Bravo Emotion 1.9CV anymore (the cce05bee-386b-11e5-a151-feff819cdc9f). We'll keep the URL</a> that identifies the resource, but we change the verb: we don't want to read (GET), we don't want to update (PUT), we want to delete (DELETE)</strong>.</p> DELETE https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f </code></pre> We don't have to supply any additional info, only the verb (DELETE) and the resource identifier.</p> Create Resources</h3> And the last verb is create (POST)</strong>. In this case we don't have to identify the resource, because it still doesn't exist.</p> POST https://cardealer.com/api/cars/ </code></pre> But we have to send the data to create the resource.</p> Following with the example, let's create a new car, so we include the necessary data within POST HTTP</a> message, it is something similar what we did at section "Update resources", but we are going to send all required data</strong>, not only the engine power.</p> JSON example:</strong></p> { "brand": "Fiat", "model": "Bravo", "year": 2010, "doors": 5, "enginePower": 120, "version": "Emotion", "clima": true, "ac": false, "fuel": "Diesel" } </code></pre> We can delegate on the system to assign a new identifier</strong>, or simply send it within the message:</p> { "identifier": "cce05bee-386b-11e5-a151-feff819cdc9f", "brand": "Fiat", "model": "Bravo", "year": 2010, "doors": 5, "enginePower": 120, "version": "Emotion", "clima": true, "ac": false, "fuel": "Diesel" } </code></pre> Collections</h3> All actions we have already explained were actually applied over a cars collection.</p> But, what happens if a resource has a nested collection?</p> Continuing with cars example, a car can use a set of engine oils. So the API</a> must allow update, delete or create elements in the set.</p> Note:</strong> For the example we will assume that the oil identifier</em> is the attribute type</em>.</p> </blockquote> Add an element to collection</h4> When we add a car to cars collection, what we do is create a new car, so it is the case of "Create Resources".</p> To add a new engine oil to the car cce05bee-386b-11e5-a151-feff819cdc9f, that already exists:</p> Request:</strong></p> POST https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/ </code></pre> Response:</strong></p> { "type": "5W30", "otherInfo": "This is the best oil for this car" } </code></pre> If we want to add another one:</p> Request:</strong></p> POST https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/ </code></pre> Response:</strong></p> { "type": "10W30", "otherInfo": "This is very good for cold weather" } </code></pre> Update a collection item</h4> If we want to update the info of oil 5W30</em> of car cce05bee-386b-11e5-a151-feff819cdc9f</em>:</p> Request:</strong></p> PUT https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/5W30/ </code></pre> Response:</strong></p> { "type": "5W30", "otherInfo": "This is no longer the best oil for this car" } </code></pre> Delete a collection item</h4> To delete an oil 10W30</em> from car cce05bee-386b-11e5-a151-feff819cdc9f</em>:</p> Request:</strong></p> DELETE https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/10W30 </code></pre> Read a collection item</h4> To get the oil info 10W30</em> of the car cce05bee-386b-11e5-a151-feff819cdc9f</em>:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/10W30 </code></pre> List collection items</h4> As we have seen at "Read a collection item", we can get the info of every collection element, but we also can get multiple collection elements, sorted, paged and apply typical collection actions.</p> We can get all supported oils for a car cce05bee-386b-11e5-a151-feff819cdc9f</em>, it is as simple as:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/ </code></pre> We can also get sorted items:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/?sort_by=type&order=asc </code></pre> We can ask API</a> to return the first 10 oils for car cce05bee-386b-11e5-a151-feff819cdc9f</em>:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/?number_of_elements=10 </code></pre> API</a> can support also pagination:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/?page=3&number_of_elements=2 </code></pre> Above request is telling API</a> that returns the page 3 of all oils of car cce05bee-386b-11e5-a151-feff819cdc9f</em> and it has to show 2 oils per page. If we want to go to next page:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/?page=4&number_of_elements=2 </code></pre> All those features are supported by query parameters</strong>.</p> Common mistake</h2> First time I tried to design a API</a> REST</a> I designed an API</a>, but not REST</a>.</p> My main mistake was the URL</a>s design, I added my own verbs</em> skipping HTTP</a> verbs</em>.</p> Wrong:</strong></p> POST https://example.com/api/cars/ford-focus/delete-oil/5W30 </code></pre> The correct way:</strong></p> DELETE https://example.com/api/cars/ford-focus/oils/5W30 </code></pre> Build C++ project with Gradle 2014-09-27T00:00:00+00:00 Introduction</h2> I am more and more worried about building, dependency management and distribution of my projects. I'd like to find a tool that unifies those processes with independence of the language. I know several tools which almost fit to what I'm looking for, like SCons</a>, Autotools</a>, Ant</a>, Maven</a> and lately Gradle</a>.</p> I've made several projects with Gradle, but always they were Java</a> and Android</a> projects. In Java projects I've found a Maven replacement, because it is faster, easier and less verbose</del>. Update 2020</strong>: With the experience, now if I have to pick a build system for a Java project I'd pick Maven, especially for a professional project, but it is not the topic of this post.</p> About Android projects I suffered the early adoption of Android Studio + Gradle</a>, although currently I think they are more mature and they work fine.</p> First of all, I have to say: building C/C++/Objective-C projects with Gradle is in incubation</a> phase, although now we can perform advanced tasks like:</p> Generation several artifacts within same project (libraries and executables).</li> Dependency management between artifacts (no versions).</li> Different "flavors" of the same software, e.g: we can generate a "Community" release and other one with more enabled features called "Enterprise".</li> It allows multi-platform binary generation.</li> </ul> As I said, this plugin is still having limitations although they are working on it: Gradle C++ roadmap</a>. If they achieve it I'll leave Autotools (I'm going to regret saying that)</del>. Update 2020</strong>: Actually few years later I am not using Autotools, neither Gradle, but I am using Meson</a> and considering Bazel</a>.</p> Case study</h2> I've extracted all the case study from Gradle user guide for native software</a>. I've adapted the project to be multi-platform with 2 versions "Community" and "Enterprise".</p> The application consists of an executable and a dynamic library. The executable will use the library.</p> Gradle also is able to generate a distributable version and a debug version.</p> You can fork the code on GitHub</a>.</p> Project Structure</h3> We can create whichever directory structure, but it is easier using the proposed by Gradle, if not we'll have to specify where the code is located.</p> This is the project structure:</p> gradle-cpp: Root directory. build.gradle: File where is configured Gradle project, it is the equivalent to: build.xml for Ant, Makefile</a> for C/C++ or pom.xml for Maven.</li> src: Folder where the source code is located. hello: This folder contains the module hello. This module will generate hello library. cpp: This folder contains C++ source files. Hello.cpp: File with the implementation of Hello class.</li> </ul> </li> headers: Folder with header files. Hello.h: Class Hello declaration.</li> Msg.h: File with constants declarations. </li> </ul> </li> </ul> </li> main: This folder contains the module which produces the executable that uses hello library. cpp: This folder contains C++ source files. main.cpp: Source code of main function. </li> </ul> </li> </ul> </li> </ul> </li> build: Folder created automatically by Gradle where it leaves all execution results like unit tests reports, compiled files, package distributions, etc.</li> </ul> </li> </ul> C++ Application</h3> It consists of an executable that uses the functionality implemented at hello</code> library.</p> main.cpp</h4> #include "Hello.h" int main(int argc, char * argv) { Hello hello ("Pepito"); hello.sayHello(10); return 0; } </code></pre> hello</code> library allows greet n</code>^{1</a></sup> times to someone who is passed as argument to constructor class.</p>} ^{1</sup> 'n' Positive integer</p> </div>}Hello.h</h4> class Hello { private: const char * who; public: Hello(const char * who); void sayHello(unsigned n = 1); }; </code></pre> Building with Gradle</h3> Base case</h4> The only we need to build the application with Gradle is: having Gradle2</a></sup> and the file build.gradle</code>.</p> ^2</sup>Actually Gradle is not required if we use the "wrapper", but we aren't going to explain it here, here you can get more info about Gradle Wrapper</a>.</p> </div> build.gradle (Base Case)</h4> apply plugin: 'cpp' model { components { hello(NativeLibrarySpec) {} main(NativeExecutableSpec) { binaries.all { lib library: "hello" } } } } </code></pre> With this simple file, we'll be able to compile and install the application in Debug mode for the platform where we are executing Gradle (in my case X64).</p> If we execute gradle task</code> from the root of the project, we'll get all the tasks we can do with Gradle.</p> In our case, we just want our compiled application ready to run, so we have to execute: gradle installMainExecutable</code>.</p> Once execution has finished, we can run the program calling to build/install/mainExecutable/main</code>3</a></sup>.</p> ^{3</sup> .bat in Windows. Without extension in Linux.</p> </div>}Output</h4> $ build/install/mainExecutable/main 1. Hello Mr. Pepito (Community) 2. Hello Mr. Pepito (Community) 3. Hello Mr. Pepito (Community) 4. Hello Mr. Pepito (Community) 5. Hello Mr. Pepito (Community) 6. Hello Mr. Pepito (Community) 7. Hello Mr. Pepito (Community) 8. Hello Mr. Pepito (Community) 9. Hello Mr. Pepito (Community) 10. Hello Mr. Pepito (Community) </code></pre> Different "Flavors"</h4> With a few lines more we can generate different versions of same application. In our example we are going to build "Community" and "Enterprise" flavors.</p> build.gradle (Flavors)</h4> apply plugin: 'cpp' model { flavors { community enterprise } components { hello(NativeLibrarySpec) { binaries.all { if (flavor == flavors.enterprise) { cppCompiler.define "ENTERPRISE" } } } main(NativeExecutableSpec) { binaries.all { lib library: "hello" } } } } </code></pre> We also have to prepare our application to use compilation parameters.</p> Msg.h</h4> #ifdef ENTERPRISE static const char * EDITION = "Enterprise"; #else static const char * EDITION = "Community"; #endif </code></pre> In this way it selects a string depending on used flavor.</p> If we execute gradle clean task</code> in the root folder, we'll get more available tasks. Before, we had installMainExecutable</code> which has been replaced by installCommunityMainExecutable</code> and installEnterpriseMainExecutable</code>.</p> If we execute both tasks, we'll get the application installed in both flavors:</p> $gradle installEnterpriseMainExecutable installCommunityMainExecutable :compileEnterpriseHelloSharedLibraryHelloCpp :linkEnterpriseHelloSharedLibrary :enterpriseHelloSharedLibrary :compileEnterpriseMainExecutableMainCpp :linkEnterpriseMainExecutable :enterpriseMainExecutable :installEnterpriseMainExecutable :compileCommunityHelloSharedLibraryHelloCpp :linkCommunityHelloSharedLibrary :communityHelloSharedLibrary :compileCommunityMainExecutableMainCpp :linkCommunityMainExecutable :communityMainExecutable :installCommunityMainExecutable BUILD SUCCESSFUL Total time: 9.414 secs </code></pre> Now we can run the application in both flavors:</p> Community</h5> $ build/install/mainExecutable/community/main 1. Hello Mr. Pepito (Community) 2. Hello Mr. Pepito (Community) 3. Hello Mr. Pepito (Community) 4. Hello Mr. Pepito (Community) 5. Hello Mr. Pepito (Community) 6. Hello Mr. Pepito (Community) 7. Hello Mr. Pepito (Community) 8. Hello Mr. Pepito (Community) 9. Hello Mr. Pepito (Community) 10. Hello Mr. Pepito (Community) </code></pre> Enterprise</h5> $ build/install/mainExecutable/enterprise/main 1. Hello Mr. Pepito (Enterprise) 2. Hello Mr. Pepito (Enterprise) 3. Hello Mr. Pepito (Enterprise) 4. Hello Mr. Pepito (Enterprise) 5. Hello Mr. Pepito (Enterprise) 6. Hello Mr. Pepito (Enterprise) 7. Hello Mr. Pepito (Enterprise) 8. Hello Mr. Pepito (Enterprise) 9. Hello Mr. Pepito (Enterprise) 10. Hello Mr. Pepito (Enterprise) </code></pre> Release or Debug</h4> By default Gradle compiles in Debug mode, but we can add the Release mode which enables several optimizations and remove debug flags^{4</a></sup>.</p>} ^{4</sup> We can also specify/modify which optimizations to apply.</p> </div>}build.gradle (Release/Debug)</h4> apply plugin: 'cpp' model { buildTypes { debug release } // ... the rest of file below doesn't change } </code></pre> If we execute gradle clean task</code> we will get more tasks, they have been split, for example installCommunityMainExecutable</code> has been split in installDebugCommunityMainExecutable</code> and installReleaseCommunityMainExecutable</code>.</p> Multi-platform</h4> Also we can use cross-compiling features provided by compilers and generate native components for other platforms. To do so we just have to add the supported platforms.</p> This only works if we have installed the Toolchain</a> for the target platform.</p> build.gradle</h4> apply plugin: 'cpp' model { buildTypes { debug release } platforms { x86 { architecture "x86" } x64 { architecture "x86_64" } itanium { architecture "ia-64" } } flavors { community enterprise } components { hello(NativeLibrarySpec) { binaries.all { if (flavor == flavors.enterprise) { cppCompiler.define "ENTERPRISE" } } } main(NativeExecutableSpec) { binaries.all { lib library: "hello" } } } } </code></pre> When execute gradle clean task</code> we get the different build options we have.</p> Note:</strong> In this example, we can build different versions of the application in different flavors for different platforms in Debug or Release mode.</p> </blockquote> Try it yourself</h3> You can find the project at https://github.com/carlosvin/cpp_gradle</a>.</p> Requirements:</p> Java 6 or higher.</li> An installed compiler (e.g GCC</a>)</li> </ul> You just have to follow next steps:</p> git clone git@github.com:carlosvin/cpp_gradle.git</code></li> cd cpp_gradle</code></li> ./gradlew task</code> or ./gradlew.bat task</code> if you are in Windows. In this way you'll see available tasks for this project. The first execution will take more time, because it downloads Gradle runtime.</li> If you are in a 64 bits platform, you can use this command to install the application: ./gradlew installX64ReleaseEnterpriseMainExecutable</code>.</li> Run the application you just built: build/install/mainExecutable/x64ReleaseEnterprise/main</code>.</li> </ol> Conclusions</h2> With a small configuration file, we can achieve many different build combinations.</p> Gradle for C++ has a promising future, and I hope it will follow the successful path of Java and Android support.</p> It is well supported by continuous integration systems and offers many plugins and features.</p> However, Gradle for C++ is still under development, so we need to be cautious:</p> Do not use it in production environments.</li> Many things may change or be removed in future versions.</li> </ul> The full example is available at https://github.com/carlosvin/cpp_gradle</a>. I encourage you to try it yourself.</p> Note:</strong> If you find any issues in this example, please leave a comment, open an issue, or submit a fix at https://github.com/carlosvin/cpp_gradle</a>.</p> </blockquote> For more information, see Getting Started with Gradle Native</a>.</p> Embedded Java Databases - Performance Comparison 2014-06-07T00:00:00+00:00 Embedded Databases</h2> These are databases that do not require a server, are embedded within the application itself, and are usually stored in local files. This, combined with the fact that they often have a mode where data is kept in memory, can result in very high performance.</p> However, this high degree of coupling to the application means they perform worse when shared between multiple applications due to access collisions.</p> Another advantage is that you don't have to maintain and manage a database server.</p> I'm going to do a performance comparison between 3 transactional embedded databases (ACID). NoSQL databases are not included in this comparison as they are in a different performance league.</p> Contents</p> SqliteDB</h3> This is a library written in ANSI C</a>, less than 500KB, multi-platform, with no external dependencies, and stores all database content in a single file.</p> It gives the best performance in the test results below.</p> It can be used from C and C++, but also from other programming languages</a> (PHP, Python, Java, .NET ...).</p> In the case of Java, we can manage this database through JDBC. The library can be obtained from here</a>.</p> Here is an example of inserting a series of objects of the class Price.java</a>:</p> public class JdbcDb implements DB { enum Column { INSTRUMENT, MARKET, PRICE, DATE; } public static final String TABLE_NAME = "PRICES"; public static final String SQL_INSERT = "INSERT INTO " + TABLE_NAME + " (INSTRUMENT, MARKET, PRICE, DATE) VALUES (?,?,?,?)"; // ... private void insert(Price p, PreparedStatement preparedStatement) throws SQLException { preparedStatement.setString(1, p.getInstrument()); preparedStatement.setString(2, p.getMarket()); preparedStatement.setDouble(3, p.getPrice()); preparedStatement.setTimestamp(4, new Timestamp(p.getDate().getTime())); preparedStatement.addBatch(); } @Override public void insert(Price... prices) throws SQLException { PreparedStatement preparedStatement = conn.prepareStatement(SQL_INSERT); for (Price p : prices) { insert(p, preparedStatement); } preparedStatement.executeBatch(); preparedStatement.close(); conn.commit(); } // ... } </code></pre> DerbyDB or JavaDB</h3> JavaDB is an Oracle distribution of the open-source DerbyDB database. It supports the ANSI/ISO SQL</a> standard via JDBC and Java EE</a>. These libraries are included in the JDK.</p> It stores the database in multiple files, which can be useful for scaling storage.</p> It can only be used in Java, not from other languages.</p> As we will see later, it was the slowest in the test results.</p> The example implementation for inserting a series of objects of the class Price.java</a> is exactly the same as above for SqliteDB. This is one of the benefits of JDBC, which allows us to obtain a connection for a specific database, but from there we can almost always forget about the specific database as long as it supports our SQL queries.</p> ObjectDB</h3> This is an object-oriented database</a> that allows access via JPA</a>, a Java standard that aims to retain the advantages of object orientation, which are often lost when dealing with persistence layers.</p> It is really easy to perform typical database actions, abstracting away from SQL. For example, here is how you would insert an array of Price objects with ObjectDB:</p> public void insert(Price... prices) throws SQLException { em.getTransaction().begin(); for (Price p : prices) { em.persist(p); } em.getTransaction().commit(); } </code></pre> If you add this ease of use to the fact that its performance in the test results was very good (close to SqliteDB), I can say that I have discovered an embedded database to consider for future Java projects. However, as with JavaDB, it is only useful if you are programming in Java.</p> The Tests</h2> The tests simply consist of repeatedly performing a series of CRUD</a> operations. For each database, the test performs a series of inserts, selects, updates, and deletions of objects of the Price class.</p> I used 100,000 instances of the Price.java</a> class, the same for each type of database.</p> For this, I created a DB.java</a> interface that each implementation for each database shares.</p> public interface DB { public void insert(Price... prices) throws SQLException; public void createTable(); public void deleteAll(); public void update(Price... prices) throws SQLException; public Set<Price> selectAll(); } </code></pre> JDBC Databases</h3> Only the data needed to obtain the driver and the database connection changes, so all the logic is in the JdbcDb.java</a> class, from which SqliteDB and DerbyDB inherit.</p> Source Code</h3> You can download the source code from here</a>.</p> It is a Gradle project, so to run the tests you just have to execute: gradle test</code>.</p> Gradle will automatically download the necessary libraries, compile, and run the tests.</p> You can also see the execution directly at Travis CI</a>.</p> </p> Test Results</h3> db.DbTest > testSqliteDB STANDARD_OUT Testing with 100000 elements 100000 Prices SqliteDB, total time 3455 ms <1> Insert: 1508 ms Select: 605 ms Update: 1319 ms Delete: 23 ms ---------------------------------------------- db.DbTest > testObjectDB STANDARD_OUT Testing with 100000 elements 100000 Prices ObjectDB, total time 6467 ms <2> Insert: 2579 ms Select: 1126 ms Update: 1698 ms Delete: 1064 ms ---------------------------------------------- db.DbTest > testDerbyDB STANDARD_OUT Testing with 100000 elements 100000 Prices DerbyDB, total time 24808 ms <3> Insert: 11467 ms Select: 695 ms Update: 6983 ms Delete: 5663 ms </code></pre> <1> SqliteDB is the fastest. <2> ObjectDB is twice as slow as SqliteDB. <3> JavaDB or DerbyDB is the slowest, about 8 times slower than SqliteDB.</p> Java serialization ways: Performance Comparison 2014-05-13T00:00:00+00:00 Recently I've had to serialize/deserialize some data in Java</a> binary format. Lately I use JSON</a> or XML</a> formats.</p> I remember that to serialize Java</a> objects they must implement the Serializable</a> interface, but I had also read in Internet other way, implementing the Externalizable</a> interface, then, which interface must I implement? It depends on what you want such as everything in the life.</p> When to use Serializable</a> or Externalizable</a>.</p> Serializable</h2> To serialize easily. You have to write less code.</li> This way has some restrictions: The object to serialize must implement the default constructor (0 args). It must be responsible to manage the parent class attributes.</li> The performance is not as important, we will see more about that in Performance tests (Serializable vs. Externalizable)</a>.</li> </ul> Externalizable</h2> You must implement the serialization/deserialization methods, so you have to write more code.</li> When you cannot use Serializable</a>.</li> When you want to improve the performance, as we'll see in Performance tests (Serializable vs. Externalizable)</a>.</li> If you have to manage the serialization of parent class attributes, then I recommend you use Externalizable</a>, because we'll avoid a weird private methods overriding.</li> </ul> private void writeObject(ObjectOutputStream oos) private void readObject(ObjectInputStream ois) </code></pre> Performance tests (Serializable vs. Externalizable)</h2> Serializable</a>: Java, through introspection, guesses the types of class attributes to know how to serialize/deserialize them, but this "magic" is not free, it has a performance penalty.</p> When we use Externalizable</a> interface, we decide how to serialize/deserialize, namely we have to write the code that does it. We've lost ease, but also we avoid that Java</a> does some tasks, so if we override the methods properly, we'll get a performance improvement.</p> To know how big is the performance difference between both interfaces, I've written a tiny example in which we serialize an object with 2 collections with 100000 elements each one</a>. Here you can find the tests execution results</a>. There are 3 different implementations:</p> Implementing Serializable</h3> As we mentioned above, Java</a> has to guess certain things. During this process it sacrifices some of performance (slowest way), in exchange we get really simple source code, the class to serialize just has to implement the Serializable</a> interface.</p> public class Contacts implements Serializable { // ... } </code></pre> serializing: 1133 millisecond</li> deserializing: 506 millisecond</li> </ul> Implementing Externalizable (wrong way)</h3> If the class implements Externalizable</a>, we must tell to Java</a> how it has to serialize/deserialize the class attributes. We just have to be careful, because if we did it bad, then we'll get the worst of the both worlds: more complex implementation and bad performance, i.e: If we serialize/deserialize complex class attributes (like collections), Java</a> will also have to guess many things about the attributes type.</p> @Override public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException { setEmails((Set<String>) in.readObject()); setPhones((Set<String>) in.readObject()); } @Override public void writeExternal(ObjectOutput out) throws IOException { out.writeObject(emails); out.writeObject(phones); } </code></pre> serializing: 737 millisecond</li> deserializing: 367 millisecond</li> </ul> Implementing Externalizable (right way)</h3> If we serialize one by one the collection elements, then we'll save more time, because Java</a> serializes simple types, this way avoids guessing things that we actually know.</p> @Override public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException { emails.clear(); phones.clear(); int nEmails = in.readInt(); for (int i = 0; i < nEmails; i++) { emails.add(in.readUTF()); } int nPhones = in.readInt(); for (int i = 0; i < nPhones; i++) { phones.add(in.readUTF()); } } @Override public void writeExternal(ObjectOutput out) throws IOException { out.writeInt(emails.size()); for (String e : emails) { out.writeUTF(e); } out.writeInt(phones.size()); for (String p : phones) { out.writeUTF(p); } } </code></pre> serializing: 204 millisecond</li> deserializing: 92 millisecond</li> </ul> We've gained performance at expense of write more code.</p> Results Analysis</h3> We don't gain performance due to use an interface or the other one.</p> We gain performance because Externalizable</a> interface forces us to implement ourselves the guessing code, so Java</a> doesn't have to do that.</p> As we have seen at Implementing Externalizable (wrong way)</a>, si no tenemos cuidado, conseguiremos una mejora poco significativa a costa de complicar nuestro código fuente.</p> Test results</a>.</li> Code in Github</a>.</li> </ul> Mutex C++ 2014-04-24T00:00:00+00:00 C++11</a> has added many improvements to help us developing multi-thread systems. I'm going to talk about Mutex</a>.</p> In previous C++11 compiler versions</a>, we can get a pthread</a> mutex, but we must initialize it and destroy it in old C style, in the end you must do more things than just lock/unlock.</p> With C++11 Mutex Class</a>, we just lock/unlock the object.</p> #include <mutex> std::mutex mtx; void do_something (int i) { mtx.lock(); // critical section mtx.unlock(); } </code></pre> Or you can just use Mutex with a generic lock guard</a>.</p> Tip:</strong> If you still have to stick to previous C++11 compiler versions, maybe it is useful to you a wrapper class I created that helps you to work with pthread mutex, so you just have to lock/unlock the Mutex object: Gist code</a>.</p> </blockquote> Android Studio Portable Download 2013-10-29T00:00:00+00:00 If you want to use Android Studio</a> without installation, as a portable software, here you have the link to last portable version of Android Studio</a>.</p> Debugging library generated with libtool 2013-02-01T15:30:00+00:00 When debugging an executable that uses a library generated with libtool</a>1</a></sup>, you might encounter the following error:</p> $ gdb ./tests-mylib "tests-mylib": not in executable format: File format not recognized </code></pre> libmylib.so</code>: is a dynamic library generated with libtool.</li> tests-mylib</code>: is an executable that uses the mylib library.</li> </ul> For those thinking I forgot to add the -g</code> compilation option, this error occurs even when using -g</code>.</p> The error is due to the fact that we are trying to execute a script generated by libtool, which is a wrapper over the actual program to facilitate its execution2</a></sup>.</p> To debug our test program tests-mylib</code></h2> libtool --mode=execute gdb tests-mylib </code></pre> ^1</sup>Tool that belongs to the Autotools</a> used to create portable software libraries.</p> </div> ^2</sup>More information in libtool documentation</a>.</p> </div> C++ Struct memory alignment 2012-11-26T00:00:00+00:00 A C++ struct is an element that groups attributes with different types so we can manipulate them all together using same reference. It is like a class with public visibility by default for functions and attributes.</p> If we want to work in a lower level, closer to machine, it might be useful to understand how that data structure is stored in memory and how to control that mapping.</p> Struct example</h2> It has two attributes: an integer (4 bytes) and a boolean (1 byte).</p> struct SampleStruct { bool flag; unsigned int timeout; }; </code></pre> If we get the instance size using sizeof</code> we should get 5 bytes size and memory would be like:</p> </p> But</strong> is not that simple, memory alignment depends on compiler and system. We will learn how to control compiler alignment policy, so we can avoid getting unexpected allocation memory sizes.</p> For example, in my local host, if I get the sizeof</code> the previous structure without pragma</code> declarations, I get a 8 bytes size</strong>. We are getting 8 Bytes instead of expected 5 Bytes because the compiler allocates more memory at the end of structure so it fits in 2n bytes blocks. Memory actually looks like:</p> </p> Example output:</p> SampleStruct flag: 1 Bytes + timeout: 4 Bytes = 5 Bytes sizeof struct: 8 Bytes -- </code></pre> TIP: If we want to know the exact structure size we have to specify compiler how to align the memory, to do so we have #pragma pack(n)</code> directive.</p> #pragma pack</code> directive in C++ struct</h2> It is a preprocessor directive to indicate to compiler how to align data in memory.</p> Example with different memory alignment configurations:</p> #pragma pack (1) struct SampleStructPack1 { bool flag; unsigned int timeout; }; #pragma pack(0) #pragma pack (2) struct SampleStructPack2 { bool flag; unsigned int timeout; }; #pragma pack(0) #pragma pack (4) struct SampleStructPack4 { bool flag; unsigned int timeout; }; #pragma pack(0) struct SampleStruct { bool flag; unsigned int timeout; }; </code></pre> Example output:</p> SampleStructPack1 flag: 1 Bytes + timeout: 4Bytes = 5Bytes sizeof struct: 5 Bytes -- SampleStructPack2 flag: 1 Bytes + timeout: 4Bytes = 5Bytes sizeof struct: 6 Bytes SampleStructPack4 flag: 1 Bytes + timeout: 4Bytes = 5Bytes sizeof struct: 8 Bytes SampleStruct flag: 1 Bytes + timeout: 4Bytes = 5Bytes sizeof struct: 8 Bytes </code></pre> SampleStructPack1 #pragma pack (1)</code>: It allocates 1 byte memory block, so our sample struct fits perfectly, in this case it is true that 4 + 1 = 5</code>.</li> SampleStructPack2 #pragma pack (2)</code>: Minimum block size is 2 bytes. Integer attribute fits because it just needs 2 blocks of 2 Bytes. Boolean attribute needs just 1 Byte, but minimum block size is 2 Bytes, that's why total allocated memory is 6 bytes, 4 + 2 = 6</code>.</li> SampleStructPack4 #pragma pack (4)</code>: It is like previous one, but in this case we are wasting more memory for boolean attribute, it needs 1 Byte, but we are allocating 4 Bytes.</li> SampleStruct (default compiler alignment): As you can see it behaves exactly like #pragma pack (4)</code>, so we can deduce it is the default compiler alignment.</li> </ol> IMPORTANT:</strong> Why don't we always use smallest memory alignment (#pragma pack (1)</code>) so we can save more memory?</p> WARNING:</strong> Because of performance loss.</p> Performance test</h2> The test will allocate same number of elements in arrays for each structure type (1, 2, 4).</p> Example output:</p> SampleStructPack1: 500000000000000000 bytes allocated in 94311 nanoseconds SampleStructPack2: 600000000000000000 bytes allocated in 1777 nanoseconds SampleStructPack4: 800000000000000000 bytes allocated in 1519 nanoseconds </code></pre> As you can see, the smallest memory alignment spends more time allocating and releasing memory.</p> Performance test source code:</p> #include <iostream> #include <chrono> #pragma pack (1) struct SampleStructPack1 { bool flag; unsigned int timeout; }; #pragma pack(0) #pragma pack (2) struct SampleStructPack2 { bool flag; unsigned int timeout; }; #pragma pack(0) #pragma pack (4) struct SampleStructPack4 { bool flag; unsigned int timeout; }; #pragma pack(0) struct SampleStruct { bool flag; unsigned int timeout; }; static const long MAX_ELEMENTS = 100000000000000000; using namespace std; using namespace std::chrono; void allocate1() { SampleStructPack1 elements [MAX_ELEMENTS]; cout << "SampleStructPack1: " << sizeof(elements) << " bytes allocated"; } void allocate2() { SampleStructPack2 elements [MAX_ELEMENTS]; cout << "SampleStructPack2: " << sizeof(elements) << " bytes allocated"; } void allocate4() { SampleStructPack4 elements [MAX_ELEMENTS]; cout << "SampleStructPack4: " << sizeof(elements) << " bytes allocated"; } void chrono1() { auto begin = high_resolution_clock::now() ; allocate1(); cout << " in " << duration_cast<nanoseconds>(high_resolution_clock::now() - begin).count() << " nanoseconds" << endl; } void chrono2() { auto begin = high_resolution_clock::now() ; allocate2(); cout << " in " << duration_cast<nanoseconds>(high_resolution_clock::now() - begin).count() << " nanoseconds" << endl; } void chrono4() { auto begin = high_resolution_clock::now() ; allocate4(); cout << " in " << duration_cast<nanoseconds>(high_resolution_clock::now() - begin).count() << " nanoseconds" << endl; } int main(int argc, char argv[]) { chrono1(); chrono2(); chrono4(); return 0; } </code></pre> Git Proxy Configuration 2012-11-15T12:00:00+00:00 Working with Git through a corporate proxy can be challenging. Here's a quick guide to configure Git proxy settings.</p> Configure HTTP Proxy</h2> git config --global http.proxy http://<username>:<password>@<ip_host>:<port> </code></pre> Configure HTTPS Proxy</h2> git config --global https.proxy https://<username>:<password>@<ip_host>:<port> </code></pre> Disable Proxy</h2> git config --global --unset http.proxy git config --global --unset https.proxy </code></pre> Parameters</h2> username</code>: Proxy username</li> password</code>: Proxy password</li> ip_host</code>: Proxy server address</li> port</code>: Proxy server port</li> </ul> Verify Configuration</h2> git config --global --get http.proxy git config --global --get https.proxy </code></pre> More secure: Environment Variables</h2> Using environment variables is a more secure method, as entering credentials directly in commands can expose them in your shell history. Instead, set your proxy credentials using environment variables:</p> export HTTP_PROXY=http://<username>:<password>@<ip_host>:<port> export HTTPS_PROXY=https://<username>:<password>@<ip_host>:<port> </code></pre> Note: Setting environment variables in the shell will also save them in the history, so this approach is more secure if you just set the environment variables in a file like ~/.bash_profile</code> (if you are using bash shell</a>).</p> For more details, see the official Git configuration documentation</a>.</p> Security Note:</strong> Storing credentials in environment variables or in files like ~/.bash_profile</code> is not inherently secure, as these files are stored in plain text and can be read by anyone with access to your home directory. For better security, consider using Git credential helpers</a> or a dedicated credential manager. If you must store sensitive data in profile files, restrict access with chmod 600 ~/.bash_profile</code> to limit who can read the file.</p> Replace punctuation symbols in Python 2012-10-23T00:00:00+00:00 Below is an explanation of how to replace punctuation symbols with whitespace in Python.</p> import re, string def remove_punctuation(text): return re.sub('[%s]' % re.escape(string.punctuation), ' ', text) </code></pre> Calling the previous function:</strong></p> >>> remove_punctuation("El perro, de San Roque, no tiene rabo; ni nunca lo ha tenido.") </code></pre> We will get this output:</strong></p> 'El perro de San Roque no tiene rabo ni nunca lo ha tenido ' </code></pre> We could make the function a little bit more generic to replace punctuation symbols with any other string.</p> import re, string def replace_punctuation(text, replace): return re.sub('[%s]' % re.escape(string.punctuation), replace, text) </code></pre> Calling the function to replace punctuation symbols by "[stop]":</strong></p> >>> replace_punctuation( "El perro, de San Roque, no tiene rabo; ni nunca lo ha tenido.", '[stop]') # output 'El perro[stop] de San Roque[stop] no tiene rabo[stop] ni nunca lo ha tenido[stop]' </code></pre> Text Normalization in Python 2012-10-02T18:00:00+00:00 In many languages, such as Spanish, there are characters that do not have ASCII</a> representation, such as á</strong>, which does have representation in Unicode</a>.</p> To avoid problems or for simplification, an equivalence has been established between Unicode</a> and ASCII</a> characters. Below I'll show you a piece of Python</a> code that performs this conversion.</p> # -- coding: utf-8 -- from unicodedata import normalize def normalize_text(text): return normalize('NFKD', text) # (1) .encode('ASCII', 'ignore') # (2) </code></pre> We specify the normal form that we apply in the normalization. In this case NFKD</code>. More information about normal form types</a>.</li> We convert the normalization result to ASCII. In case a character is erroneous, it will simply be ignored.</li> </ol> Running the function</h2> >>> normalize_text('aáaá eéeé iíií oóoó ñnñn AÀAÀ') b'aaaa eeee iiii oooo nnnn AAAA' </code></pre> Remove HTML Tags 2012-10-02T11:30:00+00:00 Below we'll see how to remove HTML tags from a character string.</p> With Python</h2> Function responsible for removing HTML tags:</p> main.py</h3> import re def strip_tags(value): return re.sub(r'<[^>]?>', '', value) </code></pre> Let's test an HTML fragment with the strip_tags</code> function:</p> Example usage</h3> import re def strip_tags(value): return re.sub(r'<[^>]*?>', '', value) html_text = """ <!DOCTYPE HTML> <html> <head> <title>Title</title> </head> <body> <p>Paragraph</p> </body> </html>""" print(strip_tags(html_text)) </code></pre> If we run the script we get the following result:</p> Title Paragraph </code></pre> If it was easy with Python</a>, let's see how it works with Django</a>.</p> Django</h2> Django</a> offers a function for this: strip_tags</code></a>.</p> First, you just need to install the library: pip install django</code>.</p> Using Django's strip_tags</h3> from django.utils.html import strip_tags html_text = """ <!DOCTYPE HTML> <html> <head> <title>Title</title> </head> <body> <p>Paragraph</p> </body> </html>""" print(strip_tags(html_text)) </code></pre> Remove orphaned packages (Linux) 2012-10-02T00:00:00+00:00 When we install a package in most Linux distributions, the package system installs other packages needed by the package that we are installing. If we uninstall the package, its dependencies might not be uninstalled; these unused dependencies are called orphaned packages.</p> Let's see how to remove orphaned packages.</p> Archlinux</h2> pacman -Rsn $(pacman -Qdtq) </code></pre> How does the command work?</p> pacman -Qdt</code> lists all orphaned packages</li> pacman -Rsn</code> uninstalls the listed packages</li> </ul> Debian</h2> apt-get remove --purge $(deborphan) </code></pre>

Building AI-Promptable Full-Stack Apps: A Reproducible Architecture

2026-03-08T00:00:00+00:00

Every time our team started a new full-stack app, we faced the same problem: rebuild the same architecture from scratch. Authentication, database access, UI shell, AI integration — all the plumbing that has nothing to do with the actual business logic.</p>

It started with internal tools, but we quickly realized the patterns apply to any full-stack web application — whether it's a customer-facing product, a dashboard, or a side project. After building several applications this way, we extracted the common patterns into a template. This post explains the architecture and the reasoning behind each decision.</p>

Live demo</a></strong> | Source code</a></strong></p> </blockquote>
The Problem</h2>
Full-stack web applications share a remarkable amount of infrastructure:</p>

A database-backed API with CRUD operations</li>
Authentication from a JWT in request headers</li>
A component library with dark/light mode</li>
Error monitoring and performance tracing</li>
Increasingly, an AI assistant that can query the data</li> </ul>
Yet every project starts from npm init</code> and rebuilds all of this. The code looks similar but is never quite the same, making it hard to maintain patterns across a growing portfolio of applications.</p>
The Stack</h2>We chose TanStack Start</a> as the foundation — a full-stack React meta-framework that gives us:</p> Server functions</strong> (createServerFn</code>) that act as type-safe RPC endpoints</li> File-based routing</strong> with TanStack Router</li> SSR</strong> via Nitro, deployable anywhere</li> Middleware</strong> that runs on every request, perfect for auth</li> </ul> For the UI, Mantine</a> gives us 120+ production-ready components, dark/light mode out of the box, and a theme system that keeps things consistent without writing custom CSS.</p> For AI, TanStack AI</a> provides a unified interface across OpenAI, Anthropic, Gemini, and more — with first-class support for tool calling and streaming.</p> Architecture: Everything Behind an Interface</h2> The core principle is simple: every external service is accessed through an interface</strong>. This makes each piece swappable without touching application code.</p> The Repository Pattern</h3> All data access goes through a ReadRepository</code> + WritableRepository</code> interface:</p> export interface ReadRepository { getTasks(filter?: TaskFilter): Promise<Task[]> getTask(taskId: string): Promise<Task | null> getAssignees(): Promise<string[]> } export interface WritableRepository { createTask(input: TaskInput, createdBy?: string): Promise<Task> updateTask(taskId: string, input: Partial<TaskInput>): Promise<Task | null> deleteTask(taskId: string): Promise<boolean> } </code></pre> Two implementations ship with the template:</p> SeedRepository</strong> — in-memory with sample data. Zero configuration, works instantly.</li> MongoRepository</strong> — production MongoDB implementation.</li> </ol> A factory function auto-detects which to use based on whether MONGODB_URI</code> is set. For development, you never need a database.</p> Auth via Global Middleware</h3> TanStack Start supports global middleware</a> — functions that run on every request before any handler executes.</p> We use this for authentication: the middleware reads the JWT from the configured header, decodes the identity, and when the user is authenticated it loads their profile from the repository. Both are attached to the request context:</p> // src/middleware/auth.ts export const authMiddleware = createMiddleware().server(async ({ next, request }) => { const authHeader = request.headers.get(AUTH_HEADER_NAME) const identity = extractIdentityFromJwt(authHeader) const user = identity.email ? identity : ANONYMOUS_USER let userProfile = null if (user.email) { userProfile = await getReadRepository().getUserProfile(user.email) } return next({ context: { user, userProfile } }) }) </code></pre> Registered globally in src/start.ts</code>:</p> export const startInstance = createStart(() => ({ requestMiddleware: [authMiddleware], })) </code></pre> Every server function automatically receives context.user</code> and context.userProfile</code> — no manual header extraction. For mutations, we chain a function-level requireAuthMiddleware</strong> (src/middleware/requireAuth.ts</code>) so only POST server functions require authentication; read-only queries stay unauthenticated. The handler can then assume context.user</code> is defined. For other cases, use the composable guards requireAuth(context)</code> and requireGroup(context, group)</code> from src/utils/auth.ts</code>; they throw with 401/403 so your handler code stays minimal.</p> Promptable by Design</h3> This is the pattern we are most excited about. AI tools call the same server functions</strong> that route loaders and event handlers use — a single code path for validation, auth, observability, and data access. Tool handlers are wrapped with withErrorHandling()</code> so that failures return { error: string, code?: number }</code> instead of crashing the agent loop:</p> const getTasksToolDef = toolDefinition({ name: 'getTasks', description: 'Get all tasks with optional filters...', inputSchema: TaskFilterSchema, // Zod schema with .describe() }) export const getTasksTool = getTasksToolDef.server((args) => withErrorHandling(() => getTasks({ data: TaskFilterSchema.parse(args) })) ) </code></pre> Because we use Zod schemas with .describe()</code> on every field, the AI model receives rich JSON Schema metadata explaining what each parameter means. The model can then compose tool calls to answer complex queries.</p> We also expose create</strong>, update</strong>, and delete</strong> as AI tools. Since these call the same server functions used by the UI — which already have requireAuthMiddleware</code> chained in — auth and creator checks happen automatically. A getCurrentUserContext</strong> tool lets the AI check who is logged in and what they can do. When the user is not allowed, mutation server functions throw with 401 or 403, withErrorHandling()</code> catches this and returns { error, code }</code>, and the AI explains it in plain language: “You need to log in to create tasks” or “Only the task creator can edit that task.”</p> The template also uses TanStack AI client tools</a> — tools that execute in the browser instead of on the server. navigate</strong> triggers router.navigate()</code> to open a page, and invalidateRouter</strong> calls router.invalidate()</code> so the UI refreshes after AI-driven mutations. Their definitions live in tools.ts</code> (shared with the server), but the implementations run in the chat drawer component via @tanstack/ai-client</code>.</p> A chat drawer talks to the backend via POST /api/chat</code> with Server-Sent Events (SSE) streaming. The client sends browser context plus current location (currentPathname</code>, currentSearch</code>, currentHref</code>), and the server injects this into the system prompt as Current Location</code> context.</p> We also keep a navigation manifest in sync with the route tree (src/services/ai/navigationManifest.ts</code>) so the model understands URL structure, including dynamic routes like /tasks/$taskId</code> (concrete URLs: /tasks/<taskId></code>). This lets the assistant interpret references such as "this page" and "this task" using the route the user is currently viewing.</p> Users get a natural language interface to their data — and to create, edit, and delete tasks when permitted — with permission-aware error handling.</p> Observability as a Plugin</h3> Monitoring is important but vendor lock-in is not. We define a minimal interface:</p> export interface ObservabilityService { startSpan<T>(name: string, fn: () => Promise<T>): Promise<T> setUser(user: { email: string; name: string }): void captureError(error: unknown): void } </code></pre> Sentry is the default implementation. If VITE_SENTRY_DSN</code> is not set, a no-op implementation is used — the app works perfectly fine without any observability configured. To switch to Datadog, New Relic, or anything else, implement the three methods.</p> Data Flow and Conventions</h3> A few conventions keep the app predictable. Loaders-first</strong>: all data is fetched in route loaders, not in useEffect</code> + useState</code>; loaders give you caching, SSR, and parallel fetching for free. URL-as-state</strong>: filters, tabs, and modal open/close live in URL search params so state is shareable, bookmarkable, and survives refresh. Mutations</strong>: POST server functions chain invalidateMiddleware</code>, so after a mutation the client automatically calls router.invalidate()</code> and refetches; components never invalidate manually. Mutation server functions throw</strong> on error; route callers wrap with processResponse()</code> for a { data, error }</code> result, so the UI can show toasts or inline errors without try/catch in every handler. E2E tested</strong>: Playwright tests run the full app against the seed repository, covering routes, filters, CRUD, and auth gating — no database needed.</p> Zod Schemas as the Single Source of Truth</h2> Every domain type starts as a Zod schema:</p> export const TaskSchema = TaskInputSchema.extend({ id: z.string().describe('Unique identifier for the task'), createdAt: z.string().describe('ISO 8601 timestamp when created'), updatedAt: z.string().describe('ISO 8601 timestamp when last updated'), }) export type Task = z.infer<typeof TaskSchema> </code></pre> This single definition serves four purposes:</p> TypeScript types</strong> — inferred via z.infer<></code></li> Server function validation</strong> — passed to .inputValidator(Schema)</code></li> AI tool schemas</strong> — converted to JSON Schema automatically</li> Documentation</strong> — .describe()</code> metadata is embedded everywhere</li> </ol> No more maintaining separate interfaces, validation logic, and documentation that inevitably drift apart.</p> Why Zod?</strong> ArkType</a> is a very good alternative and I personally prefer its syntax; we went with Zod here because it is more widely known. Zod v4 also lets you add extra metadata on fields—formatting hints, units, etc.—which is useful for formatting in the UI and for giving better hints to the AI when it uses your schemas.</p> Project structure</h3> The important pieces live under src/</code>: middleware/</code> (auth, invalidation), services/schemas/</code> (Zod as single source of truth), services/repository/</code> (interface plus seed and Mongo implementations), services/api/</code> (server functions), services/ai/</code> (adapter and tool definitions), and routes/</code> (file-based pages). The repo README has the full tree.</p> Getting Started in 30 Seconds</h2> git clone https://github.com/carlosvin/tanstack-fullstack-ai-template.git cd tanstack-fullstack-ai-template pnpm install pnpm dev </code></pre> Install The Skill (npx skills)</h2> You can install this repository's generated Agent Skill directly from GitHub:</p> # List skills available in this repository npx skills add carlosvin/tanstack-fullstack-ai-template --list # Install the TanStack fullstack pattern skill npx skills add carlosvin/tanstack-fullstack-ai-template --skill tanstack-fullstack-pattern # Optional: install globally (available across projects) npx skills add carlosvin/tanstack-fullstack-ai-template --skill tanstack-fullstack-pattern -g # Optional: verify installed skills npx skills list </code></pre> You can also try the live demo</a> without cloning.</p> For a new project you can run rm -rf .git && git init</code> after cloning to start fresh history. Use pnpm build</code>, pnpm test</code>, and pnpm lint</code> (and pnpm format</code>) to validate your fork. To run in production via Docker: docker build -t my-app .</code> then docker run --rm -p 3000:3000 my-app</code>.</p> The app starts with seed data — a working task management system with dashboard, list views, detail pages, dark/light mode, and an AI chat drawer. No database, no API keys, no environment variables needed.</p> When you're ready to add real backends, set the relevant variables. The repo includes an .env.example</code> with full documentation; the main ones are:</p> Variable</th> Purpose</th></tr></thead> MONGODB_URI</code></td> Use a real database instead of in-memory seed data</td></tr> AZURE_OPENAI_API_KEY</code>, AZURE_OPENAI_ENDPOINT</code>, AZURE_OPENAI_DEPLOYMENT</code></td> Enable the AI chat assistant (OpenAI-compatible)</td></tr> VITE_SENTRY_DSN</code></td> Enable error and performance monitoring</td></tr> AUTH_HEADER_NAME</code></td> HTTP header for the JWT (default: Authorization</code>)</td></tr> </tbody></table> Each capability layers on independently. You can clone and replace the Task domain with your own, use the generated Cursor/Windsurf skill for AI-assisted setup, or adopt layers incrementally; the repo README has the details.</p> Extending the Template</h2> Adding a new domain entity is a six-step process:</p> Zod schema</strong> in schemas.ts</code> with .describe()</code> annotations</li> Repository methods</strong> in the interface, then implement in seed and MongoDB</li> Server functions</strong> wrapping the repository (GET for loaders, POST with invalidateMiddleware</code> for mutations)</li> AI tools</strong> that call the server functions (wrapped with withErrorHandling()</code>), plus a getCurrentUserContext</strong> tool so the AI knows who is logged in and what they can do</li> Routes</strong> for the UI pages</li> Tests</strong> for the seed repository, new utilities, and E2E flows (Playwright runs against seed data)</li> </ol> Because every layer follows the same pattern, adding a new entity takes minutes, not hours. The database, AI provider, and observability layer are behind interfaces; to swap one, implement the interface and update the factory (e.g. getRepository.ts</code> for the repository, the AI adapter factory, or the observability factory in the repo).</p> Conclusion</h2> The goal is not a framework — it's a starting point</strong>. Fork the template, replace the Task domain with your own, and you have a production-ready full-stack application with:</p> Type-safe server functions</li> JWT authentication via middleware (mutations use requireAuthMiddleware)</li> AI-powered data querying and task create/edit/delete with permission-aware errors (401/403)</li> Dark/light mode UI</li> Swappable database, AI provider, and observability</li> E2E tests via Playwright against seed data (no database needed)</li> Docker-ready deployment</li> </ul> All with zero configuration for local development.</p> The template is open source at github.com/carlosvin/tanstack-fullstack-ai-template</a>. You can try the live demo</a> to see it in action before cloning. We hope it saves you the same weeks of scaffolding it saved us.</p> Built with TanStack Start, Mantine, TanStack AI, MongoDB, Zod, Sentry, Vitest, Playwright, and Biome.</em></p> Isolated Integration Tests with pytest-scenarios 2026-01-03T00:00:00+00:00 Integration tests are essential for catching bugs that unit tests miss—but they come with a notorious problem: shared state</strong>. When tests share a database, they become flaky, order-dependent, and painful to debug.</p> What if each test could start with a clean slate, with exactly the data it needs, without manual cleanup or complex fixtures?</p> That's what pytest-scenarios</code></a> does.</p> The Problem</h2> Consider testing an e-commerce checkout system. You need to verify that:</p> Active customers can complete orders</li> Suspended customers are rejected</li> Out-of-stock items fail gracefully</li> </ul> Each scenario requires different data. With traditional approaches, you either share fixtures (tests interfere), manually clean up (boilerplate hell), or use transaction rollbacks (complex and misses commit-time bugs).</p> The Solution: Declarative Test Scenarios</h2> With pytest-scenarios</code>, you declare what state you need</em>, and the plugin handles the rest:</p> def test_checkout_rejects_suspended_customer(scenario_builder, db): # Arrange: Declare the scenario scenario_builder.create({ "customers": [{"customer_id": "user_1", "status": "suspended"}], "products": [{"product_id": "item_1", "in_stock": True}], "orders": [{"id": "order_1", "customer_id": "user_1", ...}], }) # Act: Run your business logic checkout = Checkout(db) result = checkout.process(order_id="order_1") # Assert: Verify the outcome assert result.success is False assert "not active" in result.error </code></pre> No manual cleanup. No fixture dependencies. Each test is completely isolated.</p> Key Concepts</h2> Templates Define Defaults</h3> Define templates with sensible defaults—tests only override what matters:</p> # tests/templates/customers.py TEMPLATE = { "name": "John Doe", "email": "john.doe@example.test", "status": "active", } </code></pre> Automatic Cleanup</h3> The plugin clears all managed collections before each test</em>. Even if a test fails, the next test starts fresh.</p> Real Database Testing</h3> Tests run against a real MongoDB instance, catching issues that mocks would miss: query errors, index problems, schema mismatches.</p> Try It Yourself</h2> I've created a complete runnable example that demonstrates pytest-scenarios</code> with a simple e-commerce checkout system.</p> 👉 View the example on GitHub</a></strong></p> The example includes:</p> A Checkout</code> class</a> with real business logic</li> Integration tests</a> covering happy paths and edge cases</li> Templates</a> for customers, products, and orders</li> </ul> Quick Start</h3> git clone https://github.com/carlosvin/pytest-scenarios.git cd pytest-scenarios/examples </code></pre> Then follow the instructions in the example README</a> to run the tests.</p> Installation</h2> pip install pytest-scenarios # or uv add pytest-scenarios </code></pre> Configure in pyproject.toml</code>:</p> [tool.pytest.ini_options] db-url = "mongodb://127.0.0.1:27017" db-name = "my_test_db" templates-path = "tests/templates" </code></pre> Learn More</h2> 📦 PyPI Package</a></li> 📁 GitHub Repository</a></li> 📖 Example Project & README</a></li> </ul> Stop fighting flaky integration tests. Let pytest-scenarios</code> handle the isolation while you focus on what matters: testing your business logic.</p> About 2025-08-15T00:00:00+00:00 About this blog</h2> Welcome to 'My Software Engineering Journey' blog</a>, a personal blog where I document my adventures in the world of code. Here, I dive deep into topics I'm passionate about, especially within software engineering, AI, programming languages, code examples, coding practices, recent learnings and more. From deep dives into performance and efficiency to practical guides on building and deploying applications, my goal is to share what I've learned along the way. Whether you're a seasoned developer or just starting out, I hope you'll find something valuable and inspiring in these posts</a>. Join me as I explore the ever-evolving landscape of software engineering, one line of code at a time.</p> Note that there are some old posts, that are kind of legacy, I keep them for reference, one example can be Choosing a Modern C++ stack</a>, it was written in 2017, updated on 2020, but I guess today is not modern anymore 😄.</small></p> About me</h2> Before</h3> Over the years, I've had the privilege of working with industry in great companies like Microsoft</a>, eBay</a>, Champ, Santander Global Tech and Spanish Lotteries, where I've led teams, architected solutions, and developed scalable systems that power critical business functions. My expertise spans across languages and frameworks such as Go</a>, Java</a>, React</a>, Python</a>, TypeScript</a>, C++</a> and more. I've contributed to projects ranging from e-commerce platforms to real-time trading systems, always with a focus on improving efficiency, reliability, and user experience.</p> Now</h3> Currently, I lead the Sales Apps team at MongoDB</a>. Our mission is to boost the productivity of the GTM organization by building innovative software solutions. We leverage the FARM stack</a>—FastAPI</a>, React</a>, MongoDB</a>, and Next.js</a>—along with TanStack Router</a>, to create tools that help our sales teams work more efficiently and effectively.</p> We also created a sister team with the same mission of improving GTM efficiency, but focused on leveraging AI to solve problems.</p> Contact</h2> I'm always open to new opportunities and collaborations. Feel free to reach out if you'd like to connect!</p> TanStack Router: Opinionated Guidelines for Production React Apps 2025-08-15T00:00:00+00:00 This post captures a practical way to implement a React application using TanStack Router. It focuses on maintainability, predictable data flows, and ergonomics for both reading and writing code.</p> 🔗 GitHub Repository</a></li> 🚀 Live Preview</a></li> </ul> Note:</strong> The guidelines and recommendations in this guide are based on our real-world experience building a new internal application as the Sales Apps Team at MongoDB</a>. They reflect lessons learned and best practices developed throughout that process.</p> </blockquote> Chosen tech stack</h2> Typescript</h3> Faster and safer development: Our goal was to catch bugs as early as possible—ideally at compile time, together with a next level developer experience. This approach not only reduces runtime errors but also makes refactoring and onboarding significantly easier.</p> Note:</strong> In my experience, well-documented project or team guidelines have an even greater positive impact on onboarding than end-to-end typing alone.</p> </blockquote> We can see some examples of next level code assistance enabled by TypeScript in tools like Arktype</a>, TanStack Router</a>, or openapi-typescript</a>.</p> E2E type safe: Arktype + TanStack router + openapi-fetch</h3> Any input to our application will be validated and typed, there are different data inputs that can affect our application state:</p> Search params</strong>: TanStack Router offers robust, type-safe search param validation out of the box. We use Arktype</a> for runtime validation, which integrates seamlessly with TanStack Router—see this concise, powerful example</a>.</p> </li> Backend</strong>: In this case we generate a typed client from the backend OpenAPI spec</a> with openapi-fetch</code></a>. All backend interactions are handled exclusively through this generated client, ensuring type safety and consistency across the app.</p> </li> </ul> Note:</strong> For teams seeking an extra layer of runtime validation, tools like Arktype</a> can be used to validate backend responses. However, in our experience—especially as maintainers of both the backend and frontend—this has proven unnecessary. With well-maintained OpenAPI specs, semantic versioning, and strong type generation, we've yet to encounter bugs caused by a mismatch between expected and actual responses. Unless your backend is managed by a separate team or you frequently encounter contract drift, runtime validation with Arktype is likely not needed in this case.</p> </blockquote> Testing: Vitest + React testing library</h3> Here the choices are quite obvious:</p> Vitest</a> because it is the go-to testing framework if you are using Vite</a>.</li> Testing Library</a> for intuitive, user-focused component tests.</li> </ul> Project guidelines</h2> Use search params to keep page</strong> state</h3> Favor search params over component state for sharable, restorable URLs. This reduces ad‑hoc useState</code> and makes the page state linkable.</p> This guideline is one of the easiest and most impactful decisions we have made:</p> Being able for our users to share exactly the same data via URLs was really important and we get this for free.</li> If we want to change the page state, we just navigate to change search params:</li> </ul> import { createFileRoute } from '@tanstack/react-router' import { type } from 'arktype' const productSearchSchema = type({ page: 'number = 1', }) export const Route = createFileRoute('/shop/products/')({ validateSearch: productSearchSchema, loaderDeps: ({ search: { page } }) => ({ page }), loader: ({ deps: { page } }) => fetchProducts({ page }), }) function Products () { const navigate = Route.useNavigate() function handleChangePage(page: number) { // here we are just updating the query navigate({ search: (prev) => ({...prev, page})}) } } </code></pre> When the navigation is performed, the loader will be called again to fetch the products in the new page. If the route was already loaded, then it will use the cached data.</p> Page state is fully type safe, always accessible, and can be as complex as needed. For example, this URL encodes all state to list products on page 2, 100 per page, sorted by ascending price, searching "water ski": /shop/products?page=2&itemsPerPage=100&sort=asc&query=water ski&sortBy=price</code> Share this URL and everyone sees the exact same state.</p> If I want to change the state, will only have to navigate sort by date in descending order:</p> -`/shop/products?page=2&itemsPerPage=100&sort=asc&query=water ski&sortBy=price` +`/shop/products?page=2&itemsPerPage=100&sort=desc&query=water ski&sortBy=date` </code></pre> Note:</strong> Component-level state management will still be necessary for UI interactions (e.g., form inputs, toggle states, loading indicators). This guideline specifically applies to maintaining the current page state</strong>—the data and filters that define what content is displayed and should be shareable via URL.</p> </blockquote> Interacting with the backend</h3> Keep backend calls thin and predictable. Recommended approach:</p> Generate a typed client from your OpenAPI spec (e.g., with openapi-fetch</code>).</li> Wrap responses in a small helper that normalizes errors into a single AppError</code> shape your UI can consume.</li> </ul> Example shape returned by processResponse</code>:</p> type AppError = { title: string description?: string } type Processed<T> = { data?: T; error?: AppError } async function getProduct(productId: string) { // response.data: only present if 2XX response // response.error: only present if 4XX or 5XX response const response = await client.GET("/products/{product_id}", { params: { path: { product_id: productId }, }, }) // Here is where we try to convert the error into an AppError unified interface const { data, error } = processResponse({ response, action: `Fetching product ${productId}`}) return { data, error } } </code></pre> Data fetching - Loaders</h3> Leverage route loaders</a> when possible</strong> because they simplify data flows:</p> Avoid most data contexts/hooks for reads; when the route renders, data is ready to pass down</li> Built‑in caching based on declared dependencies</a>.</li> </ul> Exceptions in loaders</h4> Throw in loaders (e.g., AppError</code>) so route error components handle failures consistently. Centralize error typing so UIs can show banners/toasts with title</code> and description</code>.</p> This is just convenient, because of TanStack Error Components</a>.</p> Applying this pattern, fetching a product now throws on error for cleaner loader usage:</p> function getProduct(productId: string) { // ... same code as in the example above, the only change is below if (error) { throw error } return data } </code></pre> Mutations</h3> Mutation functions are similar to fetchers but generally should NOT throw exceptions</strong>. They're invoked from event handlers, and throwing loses error type information</strong>. Return a { data, error }</code> object instead.</p> 🚫 Ideally, we want to avoid having to inspect or discriminate error types at runtime:</p> try { throw new NotFoundError('Not Found', 404); } catch (error: unknown) { if (error instanceof NotFoundError) { console.log('The object was not found', error.objectId); } else { console.log(`Caught something else: ${error}`); } } </code></pre> ✅ Using the mutation in a route component:</p> async function handleDelete(productId: string) { const { data, error } = await deleteProduct(productId) if (error) { // The error has a title and description attributes pushToast({ variant: 'warning', ...error }) } else { pushToast({ variant: 'success', title: 'Deleted!', description: `Product ${productId}` }) // Invalidate so the loader re-fetches the page state router.invalidate() } } </code></pre> Styling</h3> Use CSS Modules for simplicity. Create Component.module.css</code> next to Component.tsx</code>. Vite handles it out of the box, no extra config needed.</p> // HomeLayout.tsx import classes from './HomeLayout.module.css' export default function HomeLayout() { return ( <div className={classes.homeContainer}> <h2 className={classes.mainTitle}>Welcome!</h2> </div> ) } </code></pre> Leverage nested routing to show nested components</h3> If a nested component isn't rendered by default (e.g., a modal), make it a nested route instead of toggling hidden UI state.</p> Benefits:</p> Clear separation of concerns; no need to render hidden components</li> Nested routes can access parent loader data</li> Lazy loading is straightforward</li> Recoverable app state via URLs (e.g., /products</code> vs /products/new</code> for opening a modal)</li> </ul> Avoid calling a loader when the parent already loaded the data</h4> In nested routes, reuse parent data:</p> const { currentUser } = useLoaderData({ from: '__root__' }) </code></pre> Leverage the default router cache</h3> The built‑in cache works well by default. After a mutation followed by navigation, you may want to router.invalidate()</code> to refetch fresh data.</p> Improve performance by setting only the needed loader deps</h4> The dependencies are also part of the caching mechanism, and only the selected search params will re-fetch data.</p> const productsIndexSchema = type({ page: 'number = 1', sort: '"newest" | "oldest" | "price" = "newest"', condensedView: 'boolean = false', }) // /routes/products.tsx export const Route = createFileRoute('/products')({ validateSearch: productsIndexSchema, loaderDeps: ({ search: { page, sort } }) => ({ page, sort }), loader: ({ deps: { page, sort } }) => fetchProducts({ page, sort, }), }) </code></pre> In this example, updating the state to display a more condensed view does not trigger a data refetch, since condensedView</code> is not included in the loaderDeps</code>. This means the loader will not re-run when only the view mode changes, optimizing performance by avoiding unnecessary network requests.</p> // This is not making the loader to re-fetch data, because condensedView is not defined in the loaderDeps Route.navigate({ search: ({prev}) => ({...prev, condensedView: true})}) </code></pre> Using router history</h3> Use history navigation when it reflects user intent (e.g., closing a modal):</p> Simpler than reconstructing navigate</code> calls with params and search state</li> Preserves cache/state more often; if you do need fresh data (e.g., post‑mutation), invalidate</li> </ul> Project structure</h3> src</code> assets</code>: public assets</li> components</code>: React components; each usually has its own folder with styles and tests</li> hooks</code>: custom React hooks</li> routes</code>: TanStack Router routes (route tree definitions and pages)</li> services</code> api</code>: wrappers around backend interactions; see fetch/mutation sections</li> schemas</code>: arktype</code> schema definitions</li> </ul> </li> </ul> </li> </ul> Easily creating a golang command line tool 2021-02-14T00:00:00+00:00 Golang</a> is one of the most useful technologies I've recently learned. Golang has pretty nice support for networking, command line, or logging out of the box—you don't need any dependency. But there are libraries making developers' life even easier.</p> I've already talked about creating REST service in go</a>, today I'd like to focus on creating a command line tool.</p> Flags</h2> Command line tool to read number of lines as an input integer parameter (Golang):</p> package main import ( "flag" "fmt" ) func main() { var nFlag = flag.Int("lines", 1234, "number of lines") flag.Parse() fmt.Printf("Lines %d\n", *nFlag) } </code></pre> With previous simple code we have already some useful capabilities:</p> Print help menu</h3> $ cmd --help > Usage of cmd: -lines int number of lines (default 1234) </code></pre> Use default value</h3> $ cmd > Lines 1234 </code></pre> Pass a value to the command tool</h3> $ cmd --lines=2 > Lines 2 </code></pre> Pass an invalid value</h3> $ cmd --lines=asdf > invalid value "asdf" for flag -lines: parse error Usage of cmd: -lines int number of lines (default 1234) exit status 2 </code></pre> For a simple command line tool, in most cases, we will have enough with the default language support. But if we want to bring more features to the combo like reading configuration from environment variables or from files, then a library like the awesome Viper</a> will come really handy.</p> Viper</h2> Viper is a very powerful and well-documented library widely used in many projects. I will not get into the details of how to use it, because I've created an abstraction to simplify its usage, I named it Meta-Viper</a>.</p> Meta-Viper</h2> Meta-Viper</a> abstracts you from the details of reading configuration from files, environment, or flags. This extra simplicity comes with a tradeoff: we are missing some flexibility.</p> Let's see some features with an example.</p> Firstly, create a go modules project with the meta-viper dependency</h3> go mod init example.com/meta-viper # Create a go modules project go get github.com/carlosvin/meta-viper # Install the meta-viper dependency </code></pre> Now let's create a program that is configurable from files, command line params, and environment variables.</p> main.go</strong></p> package main import ( "log" "os" config "github.com/carlosvin/meta-viper" ) // Here is where you define the struct that will hold the configuration values // cfg_name is the parameter name // cfg_desc is the parameter description that will be shown in the command line help type appConfig struct { Host string `cfg_name:"host" cfg_desc:"Server host"` Port int `cfg_name:"port" cfg_desc:"Server port"` SearchAPI string `cfg_name:"apis.search" cfg_desc:"Search API endpoint"` } func main() { // Instantiate the structure with default values cfg := &appConfig{ Host: "localhost", Port: 6000, SearchAPI: "google", } // Meta-Viper instance is loading the configuration from wherever is available: files, env, or input params _, err := config.New(cfg, os.Args) if err != nil { panic(err) } log.Printf("Loaded Configuration %v...", cfg) } </code></pre> Now let's see some examples of how this command line tool is able to load configuration.</p> Print usage (help)</h3> $ cmd --help Usage of flagsConfig: --apis.search string Search API endpoint (default "google") --config string Configuration name --config-dirs strings Configuration directories search paths (default [.,config,configs,cfg]) --host string Server host (default "localhost") --port int Server port (default 6000) pflag: help requested exit status 2 </code></pre> Run with default values</h3> $ cmd 2021/02/15 23:12:48 No configuration name has been specified, so no configuration file will be loaded. Using flags and environment variables. 2021/02/15 23:12:48 Loaded Configuration &{localhost 6000 google}... </code></pre> Read the port from environment variable and host from input param</h3> $ PORT=9999 cmd --host=myhost 2021/02/15 23:15:47 No configuration name has been specified, so no configuration file will be loaded. Using flags and environment variables. 2021/02/15 23:15:47 Loaded Configuration &{myhost 9999 google}... </code></pre> The last example will aggregate all the possible sources of configuration, it will extend the previous one adding configuration from a file. So let's create a configuration file:</p> the-config.json</strong></p> { "apis": { "search": "duckduckgo" } } </code></pre> Read configuration from the-config.json file, from environment and from input params</h3> $ PORT=9999 cmd --host=myhost --config=the-config 2021/02/15 23:22:17 Loaded Configuration &{myhost 9999 duckduckgo}... </code></pre> TIP: Here you can find a multi-environment example</a> a little bit more complete.</p> Parameterized Tests in Cypress 2021-01-06T00:00:00+00:00 Cypress</a> is a testing framework for anything running on a web browser. I am using it to test this site and I've talked before about it in the post "This is Sapper"</a>.</p> In this post I won't explain how to use Cypress</a>, they have a pretty nice documentation: Getting started with Cypress</a>.</p> I will stick to the parameterized tests topic.</p> Parameterized tests</strong>: Also known as dynamic tests, it is an useful technique where you can specify an input dataset and the test case will be repeated for each element in the dataset.</p> Explicit input data set</h2> Since sometimes you can’t run the test for all the possible inputs, you can select a set of meaningful data to test.</p> Example checking that the relevant pages in the site have valid HTML syntax:</p> describe('validate pages', () => { [ // <1> "/", "/categories", "/categories/sapper", "/posts/creating-custom-maven-plugin-default", "/posts/choosing-modern-cpp-stack", "/posts/cpp-mutex", "/posts/cpp-pragma-pack", "/posts/creating-custom-maven-plugin", "/posts/delete-html-tags-py-django", "/posts/gradle-cpp", "/posts/java-embedded-db-performance-comparison", "/posts/react-typescript-parcel", "/posts/pwa-lessons-learned-cra", ].forEach((url) => { // <2> it(`should be valid HTML ${url}`, () => { // <3> cy.visit(url) cy.htmlvalidate() }) }) }) </code></pre> <1> List with relevant pages to validate <2> Iterate over the list of pages <3> Test case execution for selected page</p> Test results</h3> </p> Generated input data set</h2> Instead of explicitly defining a data set, you can generate it, for example, creating a random data set.</p> Example checking that any generated random number is less than 0:</p> describe('random', () => { const inputData = Array.from(Array(100)).map(x => Math.random()) // <1> inputData.forEach(x => { // <2> it(`should be valid ${x}`, () => { // <3> expect(x).to.be.lessThan(1) }) }) }) </code></pre> <1> It generates an array of 100 random numbers <2> Iterate over the generated array <3> Test case execution</p> Test results</h3> </p> Tip:</strong> Please check the cypress official examples to learn more about dynamic/parameterized testing</a>.</p> </blockquote> Golang over Java for REST services 2020-11-01T00:00:00+00:00 Lately I've been working with two different technology stacks almost in parallel, in both cases we were using them to develop REST</a> services.</p> During this time I've come up with some conclusions and opinions I'd like to share.</p> A disclaimer, few months ago, I had several years of experience with Java and 0 days of professional experience with Golang.</p> Actual project examples</h2> Few months ago I created an API to extract and structure COVID-19 data</a> from ECDC website</a>. I developed it in Spring Boot (REST)</a>.</p> Few months later I had the luck of work on my first professional project in Go</a> and I decided to create a port of the API to extract COVID-19 data</a> in Go</a>, just for learning and for fun.</p> Now we have two REST services, almost functionally identical, but developed in two different tech stacks, so we can easily compare some relevant aspects of both.</p> Source code for the 2 REST services implementations</h3> Java</a> + Spring Boot (REST)</a></th> Go</a> + Gin framework</a></th></tr></thead> https://github.com/carlosvin/covid-rest</a></td> https://github.com/carlosvin/covid-rest-go</a></td></tr> </tbody></table> Note</strong>: I actually created that COVID-19 data REST API</a> to be the data source for the COVID19-Stats App</a>, a PWA</a> built with Svelte</a>, but that's another topic.</p> </blockquote> The Ecosystems</h2> If you want to create a REST</a> service just in plain Java you will have extra work to do, in Golang a little bit less. That's why we use framework, because they've already solved many common problems for us.</p> For this comparison I am going to use Spring Boot (REST)</a> for Java</a> and Gin framework</a> for Go</a>, but in both languages there are a lot of production ready nice options.</p> Routing</h2> Go - Without framework</h3> Go uses the concept of HTTP</a> multiplexer or router. You can specify routes using patterns and link those routes to handlers. The router will decide which handler has to execute the request based on the path received.</p> router.go file</h4> package main import ( "log" "net/http" ) func main() { router := http.NewServeMux() router.Handle("/redirect", http.RedirectHandler("https://carlosvin.github.io/", 307)) router.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) { w.Write([]byte("Hello world!")) }) log.Println("Listening...") http.ListenAndServe(":3000", router) } </code></pre> Source code is already quite simple, but there might more complex routing use cases.</p> Go - Gin Framework</h3> Happily there are frameworks that help us to keep our base code simple, for example when we need to extract path parameters, which is quite common use case in REST</a> API</a>s, we can use a routing library, I've used Gorilla Mux</a> and Gin framework</a> and I liked more Gin framework</a>.</p> Full example</a></h4> import ( "github.com/carlosvin/covid-rest-go/handlers" "github.com/carlosvin/covid-rest-go/readers" "github.com/gin-gonic/gin" ) func main() { // ... r := gin.Default() r.GET("/countries", router.Countries) r.GET("/countries/:code", router.Country) r.GET("/countries/:code/dates", router.CountryDates) r.GET("/countries/:code/dates/:date", router.CountryDate) r.Run() } </code></pre> And this is a handler example</a>, the router.Countries one</h4> func (r *routerImpl) Countries(c*gin.Context) { c.JSON(200, r.countries()) } </code></pre> Java + Spring.io</h3> The Spring Boot (REST)</a> is based on the concept of Controller, it is implemented using annotations on the class and methods.</p> Controller Example</h4> @Validated @RestController // (1) @RequestMapping("/countries") // (2) public class CountriesController { // Some source code is not shown, you can find the complete example in the repository @GetMapping("/{country}/dates/{isoDateStr}") // (3) public DateStatsDto getDateByCountry(@Size(min = 2, max = 2) @PathVariable String country, @Size(min = 10, max = 20) @PathVariable String isoDateStr) throws NotFoundException { return new DateStatsDto(service.getDate(country, DateUtils.convert(isoDateStr))); } } </code></pre> Declare the class as Controller so it is registered in Spring Boot (REST)</a></li> Controller base path definition</li> Handler definition for a nested path under the main controller path. Spring Boot (REST)</a> makes easy to extract path variables defined in the route, you can directly use them as method arguments.</li> </ol> Validations</h2> Go - Gin Framework Validation</h3> Gin framework</a> uses an external validation package validator</a>, besides that it is fully integrated with Gin framework</a>.</p> type User struct { Name string `validate:"required"` // (1) Email string `validate:"required,email"` } err := validate.Struct(user) // (2) validationErrors := err.(validator.ValidationErrors) // (3) </code></pre> The validation system uses Go</a> tags, it is not the same as Java</a> annotations, but in the validation case, it works in pretty same way as annotations.</li> Executes the validation explicitly</li> Extracts validation errors</li> </ol> Java + Spring.io Validation</h3> You can enable the validation in the controller level, then in the handlers you can also specify the type of validation. Let's explain it using the previous example:</p> CountriesController.java Validation Example</h4> @Validated // (1) @RestController @RequestMapping("/countries") public class CountriesController { // Some source code is not shown, you can find the complete example in the repository @GetMapping("/{country}/dates/{isoDateStr}") public DateStatsDto getDateByCountry( @Size(min = 2, max = 2) @PathVariable String country, // (2) @Size(min = 10, max = 20) @PathVariable String isoDateStr) throws NotFoundException { return new DateStatsDto(service.getDate(country, DateUtils.convert(isoDateStr))); } } </code></pre> Declare the class as Controller so it is registered in Spring Boot (REST)</a></li> @Size</code> validates that the input argument country has 2 characters</li> </ol> The validation system is more powerful than you can see in this code snippet, for example adding @Valid</code> annotation opens the door to complex types validation.</p> Filtering and Middleware</h3> Different approaches, pretty much the same end result.</p> I will elaborate this topic in following days.</p> Dependency injection / IoC</h2> Spring IoC</a></h3> Spring IoC</a> is the most complete and powerful systems I've ever used for IoC</a>, actually, the first time I used Spring professionally was just to deal with IoC</a>. It supports XML configuration files or Java</a> annotations, I like annotations more, here a simple example from Spring IoC</a> documentation:</p> Spring IoC example</h4> @Repository public class JpaMovieFinder implements MovieFinder { // (1) // implementation elided for clarity } // public class SimpleMovieLister { private MovieFinder movieFinder; @Autowired // (2) public void setMovieFinder(MovieFinder movieFinder) { this.movieFinder = movieFinder; } // ... } </code></pre> JpaMovieFinder</code> is instantiated by Spring IoC</a></li> With @Autowired</code> annotation Spring IoC</a> knows that has to inject movieFinder</code> argument. It should be a class implementing MovieFinder</code></li> </ol> Go</h3> Neither Go</a> nor Gin framework</a> has any IoC</a> solution, but you can still apply Dependency Injection</a> technique to decouple your components and improve the testability of your system.</p> Dependency injection simple example in Go</a></h4> package main import "fmt" // Greeter interface to greet the caller type Greeter interface { greet() } type greeterHello struct{} func (g *greeterHello) greet() { // (3) fmt.Println("Hello!") } type greeterHi struct{} func (g *greeterHi) greet() { // (4) fmt.Println("Hi!") } // App Application representation type App struct { greeters []Greeter // (1) } func (app *App) startup() { for _, v := range app.greeters { v.greet() } } func main() { greeters := []Greeter{ // (2) &greeterHello{}, &greeterHi{}, &greeterHello{}} app := &App{greeters} app.startup() } </code></pre> App</code> accepts an array of Greeter</code></li> During App</code> instantiation we pass different implementations of Greeter</code></li> Greeter implementation that prints Hello!</strong></li> Greeter implementation that prints Hi!</strong></li> </ol> It is more verbose, but there is an advantage, there is nothing hidden, everything is explicit and you have full control of instantiation order.</p> As soon as you use Dependency Injection</a>, I don't have any strong opinion about using IoC</a> system or doing Dependency Injection</a> manually.</p> Testing</h2> Unit tests</h3> For unit tests there are no big differences.</p> Go</a> comes with a standard library for testing and benchmarking</a>.</p> For Java</a> there are many well-known unit testing frameworks, but Spring already has quite big support for unit testing</a>.</p> Integration tests</h3> Go Integration Testing</h4> There are no support for Integration Tests in Go</a>, you will have to implement everything by yourself, although it is not difficult, here you can find a simple example</a>.</p> Spring</h4> On the other hand, Spring has a great testing support</a>.</p> To write integration tests for REST</a> services, MockMvc</a> is really convenient.</p> I've used MockMvc in the covid-rest project</a>.</p> MockMvc code snippet from CountriesControllerTest.java</a></h4> @Autowired private MockMvc mockMvc; // (1) @Test void getCountries() throws Exception { this.mockMvc.perform(get("/countries")) // (2) .andDo(print()).andExpect(status().isOk()) // (3) .andExpect(jsonPath("$.*", hasSize(144))) .andExpect(jsonPath("$.ES.confirmedCases",comparesEqualTo(9191))) .andExpect(jsonPath("$.ES.deathsNumber", comparesEqualTo(309))) .andExpect(jsonPath("$.ES.countryCode", comparesEqualTo("ES"))) .andExpect(jsonPath("$.ES.countryName", comparesEqualTo("Spain"))) .andExpect(jsonPath("$.ES.path", comparesEqualTo("/countries/ES"))) .andExpect(jsonPath("$.VC.countryName", comparesEqualTo("Saint Vincent and the Grenadines"))) .andDo(document("countries/list", preprocessResponse(prettyPrint(), new CropPreprocessor()))); } </code></pre> The Spring test runner injects the MockMvc object.</li> We use MockMvc to call to the endpoint we have created.</li> Then we validate the endpoint response: status code and body.</li> </ol> Performance</h2> Besides the languages specific differences, the main difference is the performance. The CPU consumption in Go</a> is smaller, but about the memory the difference is really significant, the order of 30 times smaller fingerprint</strong>.</p> Memory</h3> Here I've found a surprising difference, just by checking the memory consumption in my laptop.</p> Gin framework</a> 15.6MB</code></li> Spring Boot (REST)</a> 465.9MB</code></li> </ul> </p> Speed</h3> Following the TechEmpower benchmarks</a>:</p> Gin framework</a> is in 193 position, 9.9%.</li> Spring Boot (REST)</a> is in 284 position, 4%.</li> </ul> Following the The Benchmarker results</a>:</p> Gin framework</a>: position 33.</li> Spring Boot (REST)</a>: position 68.</li> </ul> Conclusions</h2> If I were you, I'd choose Go</a> if:</p> If you value the explicit over implicit, keep in mind that there is a cost, you will most likely have to write more lines of code.</li> If you value the simplicity, Go</a> has a quite reduced set of keywords, which reduces the learning curve and simplifies the code reviews.</li> If RAM memory usage is critical for your project, actually I'd just keep away from Spring Boot (REST)</a>.</li> If the project you are going to work on is a distributed system, specially if it is based on HTTP</a>.</li> </ul> Create a PWA with Create React App: Lessons Learned 2019-11-04T00:00:00+00:00 I started working with React</a> few year ago, always the project creation was from scratch, not using any template/scaffolding. Regarding bundlers normally I used either Webpack</a> at work or Parcel</a> for personal projects.</p> Few months ago, I wanted to start a personal project to keep track of my travel expenses. I was in a kind of rush because at that time, I was almost in the middle of my gap year, I wanted to focus on implementing main functionality and get an MVP</a> (minimum viable product) the sooner the better, so I thought it was the right time to try out Create React App</a> or CRA</a>.</p> Note</strong>: CRA</a> allows you to have a production ready PWA</a> in React</a> quickly, which is awesome. They take care of configuration and package dependencies, you only have to take care of dependencies you need for your project and of course, implement your project, CRA</a> is good, but is not magic.</p> </blockquote> As I said, I wanted to be implementing business logic ASAP, so together with using CRA</a>, I also took other decisions/shortcuts driven by the need of speeding up the development pace, I will talk about those choices in following sections describing some drawbacks and benefits.</p> Chosen Technology Stack for Budget Tracker</a></h2> CRA</a></li> React</a></li> Typescript</a></li> Material UI</a></li> Victory</a></del> Frappe charts</a></li> Firestore</a></li> Firebase Authentication</a></li> </ul> So far I am quite happy with the outcome, but with the lessons learned while developing this app, in the future, with enough time, most likely I will not choose same technology stack again</strong>. You can try the application Budget Tracker</a> and judge for yourself.</p> Along this post I will describe what are, in my experience, the benefits and drawbacks of taking these shortcuts and technical decisions.</p> Create React Application: CRA</a></h2> Create React App</a> doesn’t support Web Workers</a> neither allows to customize Service Worker</a> implementation without ejecting</a>.</p> Service worker</h3> You might want to customize your Service Worker</a> to send/receive post messages</a>, to perform background sync</a> or show web notifications</a>. In that case, you will have to eject your project</a> and maintain the configuration by yourself, which might imply a little bit of headache.</p> There are other options to customize service worker and avoid ejecting CRA</a>, but they are not straightforward enough for my taste.</p> Web worker</h3> If you need to perform any heavy processing without blocking the main thread, you can just use a Web Workers</a>, but this feature is not supported by CRA</a>. The Web Workers</a> can communicate with main thread using post messages</a> and it can also show web push notifications</a>.</p> There are also other options to use Web Workers in CRA and not eject</a>, but they imply quite some extra work.</p> Webpack</h3> Webpack</a> is the bundler used by CRA</a>. You don’t need to know much about it, unless you eject your project</a>, then you will have to deal with Webpack configuration file</a>, this is just a warning, just in case you are not comfortable with it.</p> Firebase</h2> Budget Tracker</a> supports data synchronization between different devices, so it requires a backend</a> side to deal with authentication and to save/read data remotely. I considered two options: Firebase</a> or implement REST</a> API.</p> For this backend</a>, I chose Firebase</a> because it is super easy to implement, because there is nothing to implement. You only have to configure authentication methods</a> and security rules</a> for Firestore</a>.</p> But Firebase</a> brings some drawbacks you must know before choosing it.</p> Drawbacks</h3> Bundle size</h4> I got really shocked first time I analyzed Budget Tracker</a> bundle size after integrating it with Firebase</a>, it grew around a 39%!</strong></p> 27% from Firestore</a> library.</li> 12% from Firebase Authentication</a> library.</li> </ul> Happily Budget Tracker</a> implementation is following code-splitting</a> principle, so user experience was not really affected with this integration. But user’s device will eventually have to download this extra 39% (539KB</strong>).</p> Offline first, not really</h4> Important</strong>: This section is not relevant if your use case doesn’t imply saving data linked to the user.</p> </blockquote> Firestore</a> requires user to be authenticated, but user can be anonymous</a>, this is really cool feature if you don’t want to force the user to identify to use the app.</p> Another very useful and cool Firestore</a> feature is that it supports offline mode</a>, so data can be saved and read even there is no Internet connection.</p> Anonymous user + offline mode features will allow an application to work as offline first.</p> So… what is this Offline first, not really</em> issue about? Let me explain a tricky scenario. First time the application is opened, Firebase</a> needs to authenticate the user, to do so, user’s device has to be connected to Internet, so you have to consider following scenario</strong> and either be OK with it or deal with it:</p> PWA</a> is installed in your device.</li> User is not authenticated.</li> User’s device is offline.</li> User opens the PWA</a> and tries to save some data.</li> That data won’t be saved correctly</strong>, because there is no user to link the data with, not even an anonymous user, because application needs to call Firebase</a> API to create an anonymous user.</li> </ol> This is not big deal, because it will seldom occur. If you want to deal with it anyway, check next section explaining how and why I did deal with this scenario.</p> How did I deal with this issue with Budget Tracker?</h5> First of all, this might not be an issue for your use case</strong>, because it will happen only first time application is loaded. I just wanted Budget Tracker</a> to be fully offline first, because it brings other benefits.</p> Implementation details</h6> Implement 2 persistence layers: Local (IndexedDB</a>) and Remote (Firestore</a>).</li> Save always data locally, regardless user authentication status.</li> If there is any authenticated user, after saving to local layer, propagate same action to remote layer (Firestore</a>) asynchronously.</li> </ul> Benefits</h6> If user is not authenticated, Budget Tracker</a> won’t load Firestore</a> client bundle. As I explained before, it is 27% of application size.</li> Application reads and writes are faster, because latest valid data is always saved locally. Clarification</strong>: Save data in Firestore</a> is also fast, because data is also cached locally, but it does a little bit more than just saving to IndexedDB</a> and you need an authenticated user.</li> </ul> </li> </ul> You can find a more detailed performance report</a>, where I analyze 3 different implementations:</p> Only Firestore</a> client.</li> Local (IndexedDB</a>) and remote (Firestore</a>) persistence layers.</li> Same as previous one, but remote layer implemented in service worker.</li> </ol> The performance results were in general better for option 2.</p> Data model</h4> Firestore API is easy and intuitive, I really like it, but don’t assume it will have same features as other document DBs or SQL DBs.</p> Check if Firestore limitations</a> fit into your data model, or if it is not too late, define your data model following Firestore best practices</a> and having those limitations in mind.</p> Firebase alternatives</h3> Besides implementing a REST</a> API, there are other services similar to Firebase</a> with smaller client bundle size and other features which might fit better to your requirements.</p> Consider other alternatives:</p> Based on Apache CouchDB</a>: PouchDB</a>, Cloudant</a>.</li> Based on Parse server</a>: back4apps</a>.</li> </ul> UI Components Library: Material UI</a></h2> I chose Material UI</a>.</p> Material UI</a>, from their web</p> React components for faster and easier web development. Build your own design system, or start with Material Design.</p> </blockquote> There were two reasons which drove me to use Material UI</a>:</p> To create simple UI components which are accessible, responsive</a> and with a consistent design is tricky and time consuming.</li> It has SVG set of Material Icons</a>. Budget Tracker</a> allows to create categories defined by a name and a selectable icon, so this icon set was really convenient.</li> </ul> There are some drawbacks</strong>, not very important in my opinion, maybe the most annoying for me is the first one:</p> Jest Snapshots</a> + Material UI</a>: The snapshots are generated with Material UI</a> CSS class names, but CSS classes order might not be deterministic, so a test might pass in your local host but not in CI</a> host. They are working on solve this issue, more info at github</a>.</li> Performance: There are some performance issues in Github</a>. During last months, whilst I’ve been using this library, I can say they are working hard on fix them and bring new features.</li> UI components libraries are complex and do quite a lot work, so most of them are quite heavy. Material UI bundle size weights: 304.2kB minified</a>. You can find some recommendations to reduce bundle size at Material UI website</a>.</li> </ul> Charts library</h2> Many of the chart libraries I’ve found are really powerful and complete, but they are also heavy because they depend on other third party libraries like D3</a>.</p> Initially I chose Victory</a>, but I realized that I only needed charts to show percentages and time series and Victory</a>’s bundle size is 468KB minified</a>.</p> After quick search in the Internet I discovered other lighter alternatives:</p> Frappe charts</a> 63KB minified</a>.</li> Chartist</a> 39KB minified</a>.</li> </ul> I tried them and I liked both. I chose Frappe charts</a> because I thought its default color scheme fits better with Budget Tracker</a> theme.</p> Both libraries come with more chart types than just bars and XY axis, take a quick look at their websites if you are interested about their supported chart set and to check how they look like.</p> Conclusion</h2> I will try to come up with a conclusion better than: "It depends"</em>, "Your use case will tell you"</em> and so on.</p> That said. It depends on your needs :p.</p> Seriously, let’s play "do not go for … if …"</em>:</p> Do not go for CRA if</h3> You want to customize Service Worker</a> for Background sync</a> or showing push notifications</a>.</li> You want to use Web Workers</a>.</li> </ul> Do not go for Firestore if</h3> You are aiming for your app to be hit by many users and you don’t know the estimated amount of reads/writes, otherwise you might get surprised with the bill. Firestore</a> scales like charm, maybe your budget doesn’t.</li> Bundle size is critical for your web application. Remember that bundle size is not that critical if you are implementing a PWA</a>, because your app files are cached.</li> </ul> Do not go blindly for the best charting library</h3> First of all, check what kind of charts you need. In many applications you are OK with XY axis chart, time series, bars or pie charts. You can easily get an smaller bundle size by just using a simple charting library like Frappe charts</a> or Chartist</a></p> Tip</strong>: Just check what are your requirements, if you are not sure about them, [Chosen Technology Stack for Budget Tracker</a>](#chosen-technology-stack-for-budget-tracker) consists of awesome products which most likely will fit your use case.</p> </blockquote> What next?</h2> My next technology stack bet goes for Svelte</a>/Sapper</a>, it is promising project, the results for small projects are really impressive, mainly in regards to bundle size, it is ridiculously small and development experience is quick and intuitive.</p> I’ve created a tiny PWA</a> to estimate currency exchange loss</a> when you go to a money changer shop: currency-loss.netlify.com</a>. Note, I got that app up and running in few hours, thanks to Svelte</a>.</p> Create SPA with React + Typescript + Parcel 2019-01-01T00:00:00+00:00 I was about to start yet another personal project, it consists of a SPA (Single Page Application) for a travel journal.</p> Some time ago I tried Parcel</a>, I really loved how simple it was to create a simple project from scratch, using Typescript</a> + React</a> stack. I’ve decided to create this template or base project, so next time I want to create a new SPA with my favorite frontend stack, I will only have to:</p> git clone https://github.com/carlosvin/react-typescript-parcel-template.git </code></pre> Read this before: Parcel is not as mature as Webpack</h2> If you want to create a production ready React application, use Webpack</a> or better create-react-app</a> which bring everything you need to develop a PWA</a> with React and Typescript. Following you can find an example of an app I am developing using create-react-app: https://github.com/carlosvin/budget-tracker</a>.</p> Parcel is a package bundler under development, not as mature as webpack. There are no go errors for me, at least in regards to Typescript support, see this issue in github #1378</a>.</p> I still think it is a promising project, bringing more simplicity and speed to JS bundlers world, I will give it a try again for serious projects when Parcel 2 is ready, check Parcel 2 development status</a>.</p> Quick start</h2> Development server</h3> git clone https://github.com/carlosvin/react-typescript-parcel-template.git cd react-typescript-parcel-template yarn install yarn start </code></pre> Last yarn start</code> command will:</p> start a development server at http://localhost:1234</code> with hot module replacement</a></li> build automatically development javascript files with source maps</li> </ul> Tip:</strong> Each time you save a file, you will see automatically the result at http://localhost:1234</code> without refreshing the page</p> </blockquote> Build production bundle</h3> yarn build </code></pre> Parcel’s default optimizations</a> will be applied to generated files.</p> Files are saved at dist</code> folder. Inside dist</code> folder there is also a file with information about bundle content sizes: dist/report.html</code>.</p> Step by step project creation</h2> In this section I will describe how I created this project.</p> Firstly, create package.json</code> with yarn init</a> command.</p> yarn init </code></pre> Add required dependencies</h3> Add React dependencies.</p> yarn add @types/react @types/react-dom react react-dom </code></pre> Previous command modifies package.json</code> file adding dependencies</code> section and will also install React packages in node_modules</code> folder.</p> { "name": "project_name", "version": "1.0.0", "main": "index.js", "license": "MIT", "dependencies": { "@types/react": "^16.7.18", "@types/react-dom": "^16.0.11", "react": "^16.7.0", "react-dom": "^16.7.0" } } </code></pre> Add Typescript compiler as development dependency.</p> yarn add --dev typescript </code></pre> We also need Parcel bundler</a>.</p> yarn add --dev parcel-bundler </code></pre> I’ve added a non-required dependency, it is a plugin to generate a report of generated bundle contents (the parcel version of webpack-bundle-analyzer</a>).</p> yarn add --dev parcel-plugin-bundle-visualiser </code></pre> Create application source code</h3> First we create the React application in src/index.tsx</code> file.</p> import * as React from "react"; import * as ReactDOM from "react-dom"; class App extends React.PureComponent { render() { return <h1>Hello world!</h1>; } } ReactDOM.render( <App />, document.getElementById("app") ); </code></pre> Parcel can take index.html</code> file as entry file and it figures out how to build the application, so let’s create src/index.html</code> as follows:</p> <html> <body> <div id="app"></div> <script src="./index.tsx"></script> </body> </html> </code></pre> We need div</code> tag for React to inject the DOM elements. The script</code> declaration is used by Parcel to find entry point to build.</p> Add commands build the project</h3> I’ve added the commands:</p> build</code>: Check "Build production bundle"</em> section.</li> start</code>: Check "Development server"</em> section.</li> </ul> "scripts": { "start": "parcel src/index.html", "build": "parcel build src/index.html" } </code></pre> Then to it is really easy to:</p> run development server: yarn start</code></li> generate a production bundle: yarn build</code></li> </ul> There is another approach described in Parcel documentation</a> that consists of installing Parcel globally.</p> I’ve opted for more isolated approach that affects only project you are working on, you just install Parcel as devDependency</code>. There is a tiny drawback, you can’t just run parcel index.html</code>, because it is not installed in your system, but in node_modules</code>.</p> There is a simple way to run any binary installed in node_modules</code>, you can just run npx parcel index.html</code>.</p> I like more to define build steps in package.json</code> file, so you can have well defined commands more suited to build your project. You can also use these commands as documentation how to build your project.</p> Configure Typescript (optional)</h3> Create a tsconfig.json</code> file:</p> { "compilerOptions": { "outDir": "./dist/", "sourceMap": true, "noImplicitAny": true, "module": "commonjs", "target": "es5", "jsx": "react" } } </code></pre> With this configuration, Typescript compiler will:</p> Generate files in dist</code> folder.</li> Generate source maps</a>.</li> Will not allow to declare any</code> type, for example following declaration is not allowed: const elements: any;</code></li> Generated module code will be CommonJs</a>.</li> Generated code will be ECMAScript</a> 5 compliant.</li> Support JSX</a> in .tsx</code> files.</li> </ul> Full source code</h2> You can find full example at: https://github.com/carlosvin/react-typescript-parcel-template</a>.</p> Or you can directly download the source code:</p> zip</a></li> tar.gz</a></li> </ul> Custom Maven Plugin - Override default build lifecycle 2018-05-12T00:00:00+00:00 I explained in previous article Example how to create custom Maven Plugin</a> which overrides site lifecycle</a>.</p> I have created another example to demonstrate how to override default Maven build lifecycle</a>. Default build lifecycle is used to construct your software project, for example, it is executed when you run mvn install</code> in a jar</code> type project.</p> You can find source code example at https://github.com/carlosvin/lifecycle-maven-plugin</a>.</p> Tip:</strong> I've also created an archetype</a> so you can easily play with the example.</p> </blockquote> Create an example project</h2> Generate the project using archetype</h3> mvn archetype:generate \ -DarchetypeGroupId=com.github.carlosvin.archetype \ -DartifactId=lifecycle-maven-plugin-archetype \ -DarchetypeVersion=0.6 </code></pre> Example how to create custom Maven Plugin 2018-03-11T00:00:00+00:00 Maven</a> has lots of plugins to assist you in project construction, testing, packaging and deployment. For example if you want to compile C++ code instead of Java, you can use native-maven-plugin</a>. But what if you need something more specific? Then you can create a custom Maven plugin.</p> I will explain how to create a simple custom maven plugin to generate static blog site from Markdown files. I know we can already do that with maven-site-plugin</a> since version 3.3, I will just use it for learning purposes.</p> You can find the source code of this example at https://github.com/carlosvin/blog-maven-plugin</a>.</p> Maven plugin concepts</h2> Mojo:</strong> An executable goal in Maven, e.g: mvn your-plugin:your-mojo</code> will execute a maven goal your-mojo</code> declared as part of your-plugin</code>.</p> Goal:</strong> It is equivalent to Mojo execution.</p> Lifecycle:</strong> It is a well-defined sequence of phases. Each phase consists of a sequence of goals. Let's see an example of lifecycle, e.g: FooLifecycle</code> has clean</code>, prepare</code> and assemble</code> phases. Each of those phases has one of more goals. FooLifecycle</em>:</p> clean: rmSources: a goal to remove source files</li> rmBuild: a goal to remove files in cache directory</li> </ul> </li> prepare: installDependencies: a goal to download dependencies for the project</li> </ul> </li> assemble: build: a goal to compile source files</li> </ul> </li> </ul> To define a custom life-cycle similar to previous one, we will use src/main/resources/META-INF/plexus/components.xml</code>, we will speak about that file in following sections. Normally is enough to override predefined lifecycles</a>, in this example, we will override site life-cycle</em>.</p> Tip:</strong> You can find an introduction to Maven life-cycles at Maven life-cycle guide</a></p> </blockquote> Create your custom plugin (Site Lifecycle)</h2> The plugin we are about to explain will override site lifecycle</a>, which has only 2 default phases, so when we run mvn site</code> using our new custom plugin it will execute the goals we are about to create.</p> Our plugin will work with md</code> (for Markdown</a>) file bindings: It will build and deploy the project using maven deployment plugin</a>.</p> Project structure</h3> src/main/java</code>: Where Java source code is</li> src/main/resources/META-INF/plexus/components.xml</code>: file to create/override maven lifecycles and artifact types. Here we can specify which goals will be executed when for an artifact type, for example, we can say that for an artifact of type whatever</code> when we run mvn foo</code> it will verify the files, run tests, run linter, compile and zip all generated files.</li> src/test/java</code>: Unit tests folder.</li> src/it</code>: Folder with all integration tests. Those integration tests are running actual projects and checking that outputs are as expected.</li> pom.xml</code>: File with Maven project description (Project Object Model</a>)</li> </ul> Dependency Injection</h3> Maven has finally chosen JSR-330</a> as dependency injection standard</a> (previously it was Plexus Annotations API).</p> To use dependency injection with Maven we have to:</p> Add javax.inject</code> dependency to pom.xml</code>, so we can use @Inject</code>, @Named</code>, and @Singleton</code> annotations in plugin implementation Java code.</p> pom.xml (Dependency)</h3> <dependency> <groupId>javax.inject</groupId> <artifactId>javax.inject</artifactId> <version>1</version> </dependency> </code></pre> Set up the sisu-maven-plugin</code> to index the JSR-330 components you want made available to Maven.</p> pom.xml (Plugin)</h3> <plugin> <groupId>org.eclipse.sisu</groupId> <artifactId>sisu-maven-plugin</artifactId> <version>0.3.3</version> <executions> <execution> <id>generate-index</id> <goals> <goal>main-index</goal> </goals> </execution> </executions> </plugin> </code></pre> Add annotations to your Mojo, e.g:</p> BuildMojo.java (Dependency Injection)</h3> @Mojo(name = "build", defaultPhase = LifecyclePhase.COMPILE) // <1> public class BuildMojo extends AbstractMojo { private final FileSetManager fileSetManager; private final MdToHtml mdToHtml; @Inject // <2> public BuildMojo(FileSetManager fileSetManager, MdToHtml mdToHtml) { this.fileSetManager = fileSetManager; this.mdToHtml = mdToHtml; // } } </code></pre> This annotation is not a dependency injection one, we will explain later what it is for.</li> It will inject an instance of FileSetManager and MdToHtml.</li> </ol> Write a custom Mojo</h3> It is quite straightforward to implement a Mojo class, we have to:</p> 1. Implement Mojo interface</h4> Your Mojo class has to implement org.apache.maven.plugin.Mojo</code>, although it is more convenient to extend org.apache.maven.plugin.AbstractMojo</code>, an abstract class to provide most of the infrastructure required to implement a Mojo except for execute method. That interface and class are described at Mojo API</a>.</p> public class BuildMojo extends AbstractMojo { // ... } </code></pre> 2. Configure Mojo with Java 5 annotations</h4> Annotate Mojo class with @Mojo</code> and input parameters with @Parameter</code>. Those annotations belong to another set of annotations to configure Mojos, Plugin Tools Java5 Annotations</a>.</p> /** * Generate HTML files from Markdown files */ @Mojo(name = "build", defaultPhase = LifecyclePhase.COMPILE) // <1> public class BuildMojo extends AbstractMojo { /** * Output directory path where HTML files are generated */ @Parameter( // <2> defaultValue = "${project.reporting.outputDirectory}", // <3> property = "siteOutputDirectory", // <4> required = true) private File outputDirectory; /** * A specific <code>fileSet</code> rule to select files and directories. * Fileset spec: https://maven.apache.org/shared/file-management/fileset.html */ @Parameter private FileSet inputFiles; // } </code></pre> Configures Mojo name and default life-cycle phase. To execute the Mojo in this example we will use mvn site:build</code>: site</em> is the plugin name and build</em> is name</code> parameter.</li> We use @Parameter</code> annotation to pass configuration parameters to Mojo.</li> You can use properties placeholder or any String. If the parameter type is not a String, then Maven will try to cast it.</li> It allows configuration of the Mojo parameter from the command line by referencing a system property that the user sets via the -D option. E.g: mvn site:build -DsiteOutputDirectory=/var/www/html</code> will set siteOutputDirectory attribute to /var/www/html</code>.</li> </ol> Tip:</strong> More info in Maven Plugin development guide in Parameters section</a>.</p> </blockquote> 3. Implement execute</em> method</h4> As I have explained before at 1. Implement Mojo interface</strong>, our Mojo class extends org.apache.maven.plugin.AbstractMojo</code> which has one unimplemented method from org.apache.maven.plugin.Mojo</code> interface. In that method we are going to implement the Maven goal logic.</p> Mojo class instance is called from Maven execution life-cycle by invoking execute()</code> method. Before calling execute()</code>, Maven has performed some other tasks related with the Mojo:</p> Maven instantiates Mojo and injects dependencies (see Dependency Injection</strong> section).</p> Mojo mojo = new BuildMojo(fileSetManager, mdToHtml); </code></pre> Maven configures the Mojo by assigning values to parameters.</p> I will simplify execute</code> method implementation in the sample project in github</a>, because it is more complicated and this complexity is not relevant for learning purposes.</p> Maven calls execute method: mojo.execute()</code>.</p> BuildMojo.java (Execute Method)</h3> public void execute() throws MojoExecutionException { // <1> if (inputFiles == null) { setDefaultInput(); } inputDirPath = Paths.get(inputFiles.getDirectory()); String[] includedFiles = fileSetManager.getIncludedFiles(inputFiles); // <2> outputDirPath = outputDirectory.toPath(); if (includedFiles == null || includedFiles.length == 0) { getLog().warn("SKIP: There are no input files. " + getInputFilesToString()); // <3> } else { if (!outputDirectory.exists()) { // <4> outputDirectory.mkdirs(); } try { for (String f : includedFiles) { convertToHtml(Paths.get(f), outputDirectory); // <5> } } catch (InterruptedException e) { throw new MojoExecutionException(e.getLocalizedMessage(), e); // <6> } } } </code></pre> If there is any error during execution, it should throw MojoExecutionException.</li> A way to get all selected files from FileSet</a>.</li> AbstractMojo supplies logger functionality.</li> If output directory doesn't exist, it will be created.</li> It converts each file Markdown to HTML.</li> Convert thrown exception to MojoExecutionException.</li> </ol> Unit tests</h2> In the example we use JUnit 4</a>, but you can use any other testing framework.</p> Firstly, you have to add the unit test library dependency to pom.xml</code>.</p> pom.xml</h3> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.11</version> <scope>test</scope> </dependency> </code></pre> Then you just have to write your unit tests under src/test/java</code> folder: src/test/java/com/maven/plugins/blog/PathsTest.java</a>.</p> To run the unit tests you just need to execute mvn test</code>.</p> Integration tests</h2> The 2 most popular ways to perform integration tests on custom maven plugins are using maven-failsafe-plugin</a> or maven-invoker-plugin</a>.</p> I've chosen maven-invoker-plugin because for me it is more straightforward. There is an answer at stackoverflow where they explain thoroughly the differences between them</a>.</p> How does Invoker Plugin work?</h3> We create projects to use our custom plugin under src/it</code> folder, so our plugin will be applied to test projects.</li> Invoker plugin will simulate a previously configured Maven execution.</li> After Maven execution, a validation script will check if our plugin outputs are the expected ones. For example, if our plugin is supposed to generate a file named foo.file</code>, verification plugin will check if that file exists, if it doesn't, integration test will fail.</li> </ol> Configure Invoker Plugin</h3> <plugin> <artifactId>maven-invoker-plugin</artifactId> <version>3.0.1</version> <configuration> <postBuildHookScript>verify</postBuildHookScript>  <showVersion>true</showVersion> <streamLogs>true</streamLogs> <noLog>false</noLog> <showErrors>true</showErrors> </configuration> <executions> <execution> <id>integration-test</id> <goals> <goal>install</goal>  <goal>run</goal>  </goals> </execution> </executions> </plugin> </code></pre> invoker:install</code> will be executed during the phase pre-integration-test and will install the main project artifact into target/local-repo.</li> invoker:run</code> will be executed during the integration-test phase and it will execute all defined integration tests under src/it</code> folder.</li> It configures invoker plugin to execute validation script after integration test project execution. This script may be written with either BeanShell or Groovy (verify.groovy or verify.bsh).</li> </ol> We have used other properties to show errors, show maven log and save it to a file.</p> You can check all invoker:run</code> configuration properties at maven-invoker-plugin run-mojo</a>.</p> Create an Integration Test Project</h3> It is a project we use to execute custom plugin goals, so we can validate if it produces the expected output.</p> There are 3 important files matching with AAA</a> phases ("Arrange-Act-Assert").</p> Arrange: pom.xml</h4> This file is a project using our custom plugin.</p> src/it/md-html/pom.xml</a></p> <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>com.maven.plugins.it</groupId> <artifactId>simple-it</artifactId> <version>1.0-SNAPSHOT</version> <build> <plugins> <plugin> <groupId>@project.groupId@</groupId> <artifactId>@project.artifactId@</artifactId> <version>@project.version@</version> </plugin> </plugins> </build> </project> </code></pre> It is a very simple pom file where we use placeholders to reference to our plugin under test. When invoker plugin executes following pom file, firstly will replace those placeholders to reference to the latest version of our custom plugin which was recently installed in the local repository:</p> <plugin> <groupId>com.maven.plugins</groupId> <artifactId>blog</artifactId> <version>0.0.1-SNAPSHOT</version> </plugin> </code></pre> In that way invoker plugin ensures it is testing the latest version of current project.</p> Act: invoker.properties</h4> It configures how test project will be executed.</p> src/it/md-html/invoker.properties</a></p> invoker.goals = blog:build invoker.name = Test build MD </code></pre> It will execute mvn blog:build</code>, a goal defined in our custom plugin under example or what is the same, it will execute BuildMojo described in section Write a custom Mojo</strong>.</p> Assert: verify.groovy</h4> It is the script to check that plugin execution generated the expected results.</p> Verification script, it is checking if target/site/README.html</code> file was generated by the plugin.</p> src/it/md-html/verify.groovy</a></p> File generated = new File( basedir, "target/site/README.html" ); assert generated.isFile() </code></pre> Choosing a Modern C++ stack 2017-09-15T00:00:00+00:00 I'm starting a new project in C++, but I've run into a couple of questions before starting:</p> Which build system should I use?</li> Which unit testing framework?</li> </ol> Tip</strong>: If you just want a project template so you can have a C++ project skeleton ready in seconds, just go to uuid-cpp</a> and follow the instructions in README.md</a>.</p> </blockquote> Choosing Build System (Meson</a>)</h2> I have used before Make</a>, Maven</a>, Scons</a>, Gradle</a> and Autotools</a>, but I have some reasons to try something else:</p> Autotools:</strong> It is not easy to configure and maintain. There are several configuration files and several configuration steps.</li> Gradle:</strong> CPP feature is still incubating. Not very fast. You can check a similar example project at Build C++ project with Gradle</a>.</li> Make:</strong> I don't love the syntax. Files tend to get messy as the project grows.</li> Scons:</strong> It is just slower and not as easy to understand as Meson.</li> Maven:</strong> It is slow and you might end up "Javatizing" your C++ project structure.</li> </ul> Note</strong>: I've listed just things I don't like; those projects have other great features.</p> </blockquote> CMake</a> vs Meson</a></h3> After discarding previous ones, I'm considering Meson or CMake. Both are fast build systems:</p> Although Meson is written in Python</a>, it generates a Ninja</a> build project. The first time you configure the project you have to run Meson, but for building or testing you are actually running Ninja.</p> CMake can also generate Ninja files among other formats; check CMake generators documentation for more information</a>.</p> CMake:</strong> It has a big advantage over Meson; it is mature and widely used in many projects, which means there are many examples and it will fulfill your C++ project building needs.</li> Meson:</strong> It is a young project compared with CMake, but it is growing quite fast and it has been adopted in other big projects like Gnome</a>, which has an initiative to port from Autotools to Meson</a>.</li> </ul> Finally, I've chosen</strong> Meson because syntax is really clear to me; when I read a meson.build</code> file I can quickly understand what is happening during the build process.</p> Steps to compile and test a project</h3> meson build . # First time you configure the project cd build ninja build # Each time you build it ninja test # Each time you run tests </code></pre> First time you configure the project</li> Each time you build it</li> Each time you run tests</li> </ol> Other build systems comparisons</h3> I've found two interesting comparisons about available C++ build systems; they might be a little biased because those comparisons come from Meson and Scons.</p> C++ build systems comparison from Scons</a></li> C++ build systems comparison from Meson</a></li> </ul> Unit Testing Framework</h2> I have used some xUnit</a>-based libraries like UnitTest++</a>, CppUTest</a> or Google Test</a>, which pair perfectly with Google Mock</a>. If you want a safe bet that fulfills almost all of your testing needs I highly recommend Google Test.</p> But some time ago I found a testing framework with some interesting features: Catch</a>:</p> It is just a header file with no external dependencies, so very easy to start (wget + include downloaded file).</li> You can use normal unit test style or BDD</a>-style.</li> </ul> If you want to know more about Catch, I recommend you to give it a try; it takes just 2 minutes to have a simple example up and running</a>. You can also read some interesting articles like Why do we need yet another C++ test framework?</a> or Testing C++ With A New Catch</a>.</p> Doctest</a>: A Catch alternative</h3> There is another testing framework named Doctest, with the same benefits as Catch, but it promises to be faster and lighter (performance results</a>) than Catch.</p> Doctest is modeled after Catch and some parts of the code have been taken directly, but there are differences</a>.</p> It hasn't been easy to decide; both are really similar. Below you can see some differences:</p> @@ -1,12 +1,12 @@ -#define CATCH_CONFIG_MAIN // It tells Catch to provide a main() - only do this in one cpp file +#define DOCTEST_CONFIG_IMPLEMENT_WITH_MAIN -#include "catch.hpp" +#include "doctest.h" #include "Uuid.h" #include <string> constexpr int MAX_ITERS = 100; -TEST_CASE("Uuid", "[uuid]") +TEST_CASE("Uuid") @@ -26,7 +26,7 @@ TEST_CASE("Uuid", "[uuid]") // BDD style -SCENARIO("UUID creation", "[Uuid]") +SCENARIO("UUID creation") </code></pre> I've finally chosen Doctest because it promises to be faster: performance results</a>.</p> Note</strong>: I've created the project using both frameworks; you can find them in the corresponding branches: doctest branch</a> or catch branch</a>.</p> </blockquote> Example</h2> I've created an example to illustrate this article: uuid-cpp</a>. It is a basic implementation of UUID pseudo-random generator based on mt19937</a>, which is not cryptographically secure.</p> Project output artifacts</h3> When we install the project using Meson (Ninja), we will get some artifacts generated and copied in our system:</p> Shared library: libuuid</code>.</li> Header library for developers who want to use the shared library: include/Uuid.h</code>.</li> Executable uuidgen</code> (UUID</a> generator).</li> Test executable (not installed). It tests the shared library.</li> </ul> For example, if you execute:</p> /usr/local/lib/libuuid.so /usr/local/include/Uuid.h /usr/local/bin/uuidgen </code></pre> Project structure (Fork project</a>)</h3> ...existing code...</p> Filesystem in C++17 2017-05-28T00:00:00+00:00 Introduction</h2> Since C++17 new filesystem abstractions will be added to C++ environment. So far they are available as Experimental C++ Features</a>. If you want to dig more about this new library, here it is the final draft of File System Technical Specification</a>.</p> Gettting started with Experimental Filesystem Features C++17 (g++)</h2> We just have to "tell" compiler that:</p> we write C++17 (-c++1z</code>) and</li> it has to add standard library with filesystem library</em> (-lstdc++fs</code>).</li> </ul> g++ -std=c++1z main.cpp -lstdc++fs && ./a.out </code></pre> Let's see a simple example with std::filesystem::path</code> class.</p> #include <experimental/filesystem> #include <iostream> namespace fs = std::experimental::filesystem; using namespace std; int main() { fs::path aPath {"./path/to/file.txt"}; cout << "Parent path: " << aPath.parent_path() << endl; cout << "Filename: " << aPath.filename() << endl; cout << "Extension: " << aPath.extension() << endl; return 0; } </code></pre> Compile and run: Basic C++17 example</a></h3> $ g++ -std=c++1z main.cpp -lstdc++fs && ./a.out $ ./a.out Parent path: "./path/to" Filename: "file.txt" Extension: ".txt" </code></pre> C++17 Filesystem Features</h2> In this section, we are going to explain some std::filesystem</a> features with examples, which will help us to highlight differences between C++11 and C++17 so we can get a better idea about what this new library will supply and how it might make developer's work easier.</p> std::filesystem::path</h3> Upper we have seen a tiny use case for std::filesystem::path</a>. That is a quite powerful and convenient feature that supplies an multi-platform abstraction for paths to files using the correct directory path separator depending on the platform we are building our application for (\</code> for Windows based systems and /</code> for Unix based systems).</p> Directory separator</h3> When we want our application to use the correct directory separator in C++11, we could use conditional macro declaration:</p> Platform independent directory separator in C++11</h4> #include <iostream> using namespace std; #ifdef _WIN32 const string SEP = "\\"; #else const string SEP = "/"; #endif int main() { cout << "Separator in my system " << SEP << endl; return 0; } </code></pre> Compile and run: C++11 separator example</a>.</p> Platform independent directory separator in C++17. Cleaner and simpler</h4> #include <experimental/filesystem> #include <iostream> namespace fs = std::experimental::filesystem; using namespace std; int main() { cout << "Separator in my system " << fs::path::preferred_separator << endl; return 0; } </code></pre> Compile and run: C++17 separator example</a>.</p> Directory Separator Operator</h3> std::filesystem::path</a> implements /</code> operator, which allows to easily concatenate paths to files and directories.</p> When we want to concatenate paths in C++11, we have to add extra logic to avoid adding duplicate separators and to select the correct separator for target platform:</p> Concatenate paths in C++11</h4> #include <iostream> using namespace std; #ifdef _WIN32 const string SEP = "\\"; #else const string SEP = "/"; #endif int main() { string root {"/"}; string dir {"var/www/"}; string index {"index.html"}; string pathToIndex{}; pathToIndex.append(root).append(SEP).append(dir).append(SEP).append(index); cout << pathToIndex << endl; return 0; } </code></pre> Compile and run: Concatenate paths in C++11</a></h4> //var/www//index.html </code></pre> Checking program output we notice it is not fully correct, we should have checked whether path parts already contains a separator so we don't append another separator again. That logic is already implemented in std::filesystem::path</a>, so C++17 can be like:</p> Concatenate paths in C++17</h4> #include <experimental/filesystem> #include <iostream> namespace fs = std::experimental::filesystem; using namespace std; int main() { fs::path root {"/"}; fs::path dir {"var/www/"}; fs::path index {"index.html"}; fs::path pathToIndex = root / dir / index; cout << pathToIndex << endl; return 0; } </code></pre> Compile and run: Concatenate paths in C++17</a></h4> "/var/www/index.html" </code></pre> Code is cleaner and just correct, there are no duplicated separators.</p> Create/Remove Directories</h3> std::filesystem</a> comes with some utilities to create and remove files and directories, but firstly let's try to do so in C++11.</p> Create and remove nested directories in C++11</h4> #include <iostream> #include <cstdio> #include <sys/stat.h> using namespace std; int main() { auto opts = S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH; mkdir("sandbox", opts); mkdir("sandbox/a", opts); mkdir("sandbox/a/b", opts); mkdir("sandbox/c", opts); mkdir("sandbox/c/d", opts); system("ls -la sandbox/*"); remove("sandbox/c/d"); remove("sandbox/a/b"); remove("sandbox/c"); remove("sandbox/a"); remove("sandbox"); system("ls -la"); return 0; } </code></pre> Compile and run: Create and remove directories C++11</a></h4> g++-4.9 -std=c++11 main.cpp -lm && ./a.out sandbox/a: total 12 drwxr-xr-x 3 2001 2000 4096 May 28 12:27 . drwxr-xr-x 4 2001 2000 4096 May 28 12:27 .. drwxr-xr-x 2 2001 2000 4096 May 28 12:27 b sandbox/c: total 12 drwxr-xr-x 3 2001 2000 4096 May 28 12:27 . drwxr-xr-x 4 2001 2000 4096 May 28 12:27 .. drwxr-xr-x 2 2001 2000 4096 May 28 12:27 d total 8012 drwxrwxrwx 2 2001 2000 4096 May 28 12:27 . drwxrwxrwx 3 2002 2000 8175616 May 28 12:27 .. -rwxr-xr-x 1 2001 2000 8168 May 28 12:27 a.out -rw-rw-rw- 1 2001 2000 517 May 28 12:27 main.cpp </code></pre> We have to create/remove one by one. We could rewrite this code snippet with less lines (using a loop), but we still have to pay attention to creation/deletion order, we cannot remove parent directory before we have removed all children.</p> Since C++17, we can create and remove nested directories with just one call.</p> Create and remove nested directories C++17</h4> #include <experimental/filesystem> #include <iostream> namespace fs = std::experimental::filesystem; using namespace std; int main() { fs::create_directories("sandbox/a/b"); fs::create_directories("sandbox/c/d"); system("ls -la sandbox/*"); cout << "Were directories removed? " << fs::remove_all("sandbox") << endl; system("ls -la"); return 0; } </code></pre> Compile and run: Create and remove nested directories C++17</a></h4> g++ -std=c++1z -fconcepts -fgnu-tm -O2 -Wall -Wextra -pedantic -pthread -pedantic-errors main.cpp -lm -latomic -lstdc++fs && ./a.out sandbox/a: total 12 drwxr-xr-x 3 2001 2000 4096 May 28 16:45 . drwxr-xr-x 4 2001 2000 4096 May 28 16:45 .. drwxr-xr-x 2 2001 2000 4096 May 28 16:45 b sandbox/c: total 12 drwxr-xr-x 3 2001 2000 4096 May 28 16:45 . drwxr-xr-x 4 2001 2000 4096 May 28 16:45 .. drwxr-xr-x 2 2001 2000 4096 May 28 16:45 d Were directories removed? 1 total 10132 drwxrwxrwx 2 2001 2000 4096 May 28 16:45 . drwxrwxrwx 3 2002 2000 8175616 May 28 16:45 .. -rwxr-xr-x 1 2001 2000 2170976 May 28 16:45 a.out -rw-rw-rw- 1 2001 2000 393 May 28 16:45 main.cpp </code></pre> Full example: Recursive Directory Iterator</h2> This example consists of iterate recursively through dicrectories fintering files by extension.</p> C++11</h3> To keep C++11 example simple, I haven't added filtering logic, but filtering logic is present in C++17 example:</p> filesystem.11.cpp</h4> #include <dirent.h> #include <cstring> #include <iostream> #include <fstream> // std::ofstream #include <vector> #include <memory> #include <system_error> #include <sys/stat.h> using namespace std; const string UP_DIR = ".."; const string CURRENT_DIR = "."; const string SEP = "/"; string path(initializer_list<string> parts) { string pathTmp {}; string separator = ""; for (auto & part: parts) { pathTmp.append(separator).append(part); separator = SEP; } return pathTmp; } vector<string> getDirectoryFiles(const string& dir, const vector<string> & extensions) { vector<string> files; shared_ptr<DIR> directory_ptr(opendir(dir.c_str()), [](DIR* dir){ dir && closedir(dir); }); if (!directory_ptr) { throw system_error(error_code(errno, system_category()), "Error opening : " + dir); } struct dirent *dirent_ptr; while ((dirent_ptr = readdir(directory_ptr.get())) != nullptr) { const string fileName {dirent_ptr->d_name}; if (dirent_ptr->d_type == DT_DIR) { if (CURRENT_DIR != fileName && UP_DIR != fileName) { auto subFiles = getDirectoryFiles(path({dir, fileName}), extensions); files.insert(end(files), begin(subFiles), end(subFiles)); } } else if (dirent_ptr->d_type == DT_REG) { // here we should check also if filename has an extension in extensions vector files.push_back(path({dir, fileName})); } } return files; } int main () { auto opt = S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH; mkdir("sandbox", opt); mkdir("sandbox/a", opt); mkdir("sandbox/a/b", opt); vector<string> e_files = { "./sandbox/a/b/file1.rst", "./sandbox/a/b/file1.txt", "./sandbox/a/file2.RST", "./sandbox/file3.md", "./sandbox/will_be.ignored" }; // create files for (auto &f: e_files) { ofstream of(f, ofstream::out); of << "test"; } cout << "filtered files: " << endl; for (auto &f: getDirectoryFiles(".", {".rst", ".RST", ".md"})){ cout << "\t" << f << endl; } return 0; } </code></pre> Compile and run C++11 example</a></h4> g++ -std=c++11 -O2 -Wall -Wextra -pedantic -pthread -pedantic-errors main.cpp -lm -latomic -lstdc++fs && ./a.out filtered files: ./main.cpp ./sandbox/file3.md ./sandbox/will_be.ignored ./sandbox/a/b/file1.rst ./sandbox/a/b/file1.txt ./sandbox/a/file2.RST ./a.out </code></pre> C++17</h3> Following example also filters files by extension.</p> filesystem.17.cpp</h4> #include <dirent.h> #include <cstring> #include <iostream> #include <fstream> // std::ofstream #include <vector> #include <memory> #include <system_error> #include <sys/stat.h> using namespace std; const string UP_DIR = ".."; const string CURRENT_DIR = "."; const string SEP = "/"; string path(initializer_list<string> parts) { string pathTmp {}; string separator = ""; for (auto & part: parts) { pathTmp.append(separator).append(part); separator = SEP; } return pathTmp; } vector<string> getDirectoryFiles(const string& dir, const vector<string> & extensions) { vector<string> files; shared_ptr<DIR> directory_ptr(opendir(dir.c_str()), [](DIR* dir){ dir && closedir(dir); }); if (!directory_ptr) { throw system_error(error_code(errno, system_category()), "Error opening : " + dir); } struct dirent *dirent_ptr; while ((dirent_ptr = readdir(directory_ptr.get())) != nullptr) { const string fileName {dirent_ptr->d_name}; if (dirent_ptr->d_type == DT_DIR) { if (CURRENT_DIR != fileName && UP_DIR != fileName) { auto subFiles = getDirectoryFiles(path({dir, fileName}), extensions); files.insert(end(files), begin(subFiles), end(subFiles)); } } else if (dirent_ptr->d_type == DT_REG) { // here we should check also if filename has an extension in extensions vector files.push_back(path({dir, fileName})); } } return files; } int main () { auto opt = S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH; mkdir("sandbox", opt); mkdir("sandbox/a", opt); mkdir("sandbox/a/b", opt); vector<string> e_files = { "./sandbox/a/b/file1.rst", "./sandbox/a/b/file1.txt", "./sandbox/a/file2.RST", "./sandbox/file3.md", "./sandbox/will_be.ignored" }; // create files for (auto &f: e_files) { ofstream of(f, ofstream::out); of << "test"; } cout << "filtered files: " << endl; for (auto &f: getDirectoryFiles(".", {".rst", ".RST", ".md"})){ cout << "\t" << f << endl; } return 0; } </code></pre> Compile and run C++17 example</a></h4> g++ -std=c++11 -O2 -Wall -Wextra -pedantic -pthread -pedantic-errors main.cpp -lm -latomic -lstdc++fs && ./a.out filtered files: ./main.cpp ./sandbox/file3.md ./sandbox/will_be.ignored ./sandbox/a/b/file1.rst ./sandbox/a/b/file1.txt ./sandbox/a/file2.RST ./a.out </code></pre> Multi-Domain Docker Containers 2016-11-24T00:00:00+00:00 Use case</h2> We have several server applications in the same development environment, each application is bundled in a Docker container, e.g: "Container A"</strong> and "Container B"</strong>.</p> With Docker those applications have the same IP address. One way to differentiate and access to an specific application is exposing different ports.</p> </p> If we want to call to "Application A"</strong> we will do: GET http://10.20.30.40:8080/colors/red</code>.</li> If we want to call to "Application B"</strong> we will do: GET http://10.20.30.40:8081/fruits/tomato</code>.</li> </ul> But that solution is a little bit confusing, does 8080 mean we are accessing to "application A"?</p> It would be simpler and easier</strong> to remind something like:</p> Calling "Application A"</strong>: GET http://a.domain.com/colors/red</code>.</li> Calling "Application B"</strong>: GET http://b.domain.com/fruits/tomato</code>.</li> </ul> </p> Get that extra semantic value is much simpler than I thought at the beginning and you will see below.</p> How to Configure Multi-Domain Reverse Proxy</h2> I said it is easy, because we almost have to do nothing, another container will do it for us, especifically we are going to use nginx-proxy</a>, it will automatically generate the required NGINX</a> configurations.</p> So, we will have 2 applications + 1 proxy, that is 3 containers.</p> </p> Note</strong>: You can download the full example at https://github.com/carlosvin/docker-reverse-proxy-multi-domain</a>.</p> </blockquote> Example Project Structure</h3> docker-compose.yaml</strong>: Main configuration file describing architecture in previous picture.</li> a</strong>: Application A directory. Dockerfile</strong>: Container A configuration file.</li> </ul> </li> b</strong>: Application B directory. Dockerfile</strong>: Container B configuration file.</li> </ul> </li> </ul> View Project</a></p> Architecture Configuration (docker-compose)</h3> The relationships between containers is the most interesting part in this example.</p> docker-reverse-proxy-multi-domain/docker-compose.yaml</h3> a: build: a # (4) environment: VIRTUAL_HOST: a.domain.com # (1) restart: always b: build: b # (5) environment: VIRTUAL_HOST: b.domain.com # (2) restart: always nginx-proxy: # (3) image: jwilder/nginx-proxy ports: - "80:80" - "443:443" volumes: - /var/run/docker.sock:/tmp/docker.sock:ro restart: always privileged: true </code></pre> Configure the domain name for app a.</li> Configure the domain name for app b.</li> From this line there is proxy configuration (copy/paste part).</li> We tell docker-compose has to build Docker images within specified directory.</li> For example, we are saying that docker-compose has to build a Docker image using ../b/Dockerfile file.</li> </ol> Application Image Configuration</h3> a/Dockerfile</h3> FROM httpd:2.4 # (1) RUN echo "<html><body><h1>A</h1>App A works!</body></html>" > /usr/local/apache2/htdocs/index.html # (2) </code></pre> We import an image with an apache server.</li> It serves a file that prints "Host A" as default page.</li> </ol> The configuration for application B is pretty much the same:</p> b/Dockerfile</h3> FROM httpd:2.4 RUN echo "<html><body><h1>B</h1>App B works!</body></html>" > /usr/local/apache2/htdocs/index.html </code></pre> Adding domain names to your development environment configuration</h3> In Linux we just have to map the local address to domain names you have chosen, in the example a.domain.com</code> and b.domain.com</code>.</p> /etc/hosts</h3> 127.0.0.1 localhost.localdomain localhost ::1 localhost6.localdomain6 localhost6 127.0.0.1 a.domain.com # (1) 127.0.0.1 b.domain.com </code></pre> We just added last 2 lines.</li> </ol> Everything ready</h3> Now we just have to test the example:</p> docker-compose build docker-compose up </code></pre> The 3 containers are running now.</p> So we can open our favorite web browser and go to a.domain.com</code>. It will show App A works!</strong>. If we go to b.domain.com</code> then we will see App B works!</strong>.</p> </p> </p> Note</strong>: In most of the Linux distros you will need privileges to run Docker commands (sudo</code>).</p> </blockquote> Convert files formats - Windows to Unix 2016-02-12T00:00:00+00:00 If you are developing from a Windows environment to a Unix target environment, most likely you have had this issue: You install source files in Windows format in your Unix environment.</p> Windows and Unix systems use different line endings: Windows uses carriage return and line feed (\r\n</code>), while Unix uses just line feed (\n</code>). This difference can cause issues with scripts, code, and configuration files when moving files between systems.</p> There is a quite simple way to convert all your files from Windows to Unix format:</p> find . -type f -print0 | xargs -0 dos2unix </code></pre> I got it, of course, from this Stack Overflow answer</a>.</p> REST URLs 2015-08-16T00:00:00+00:00 First time I designed a REST</a> API</a> I made several mistakes, of course. Following I'm going to explain common mistakes and what I've learned about REST</a> URL</a> with examples.</p> REST Basics</h2> Using URL</a>s for get resources.</li> Using verbs</em> for modify resources.</li> The verbs</em> are provided by the HTTP</a> protocol.</li> The verbs</em> have a direct equivalency with CRUD</a> (Create, Read, Update, Delete).</li> To access to an existent resource we need an identifier.</li> </ul> REST Verbs</h3> POST: Create</strong> new resources. GET: Read</strong> already existing resources. PUT: Update</strong> already existing resources. DELETE: Delete</strong> already existing resources.</p> It is clearer in the following table:</p> REST Verb</th> CRUD Action</th> Resource must exist</th></tr></thead> POST</td> Create</td> No</td></tr> GET</td> Read</td> Yes</td></tr> PUT</td> Update</td> Yes</td></tr> DELETE</td> Delete</td> Yes</td></tr> </tbody></table> Accessing to Resources</h3> A resource is what we want to get</em>. For example, a car.</p> To be able to get a car, that information is not enough, you can't go to your car dealer and ask for whatever car, you have to specify which one you want:</p> Good morning. I'd like to have a Fiat Bravo 1.9 Emotion 120CV.</p> </blockquote> In this manner the seller knows which one is.</p> "Fiat Bravo 1.9 Emotion 120CV" is the identifier</strong>.</p> Transferring the example to REST</a> API</a>s:</p> GET https://cardealer.com/api/cars/fiat-bravo-19-emotion-120cv </code></pre> Now our API</a> can supply the car info.</p> This is a very simple example, but actually when we access to a specific resource, we have to use something to identify it, a common and recommendable practice is use UUID</a>.</p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f </code></pre> But our API</a>, like a shop, it hasn't to be so strict. We can ask for cars with several features:</p> Good morning, I want a Fiat Bravo.</p> </blockquote> Then, the dealer kindly will show you all Fiat Bravo he has available. Let's see how API</a> says that.</p> GET https://cardealer.com/api/cars/?brand=fiat&model=bravo </code></pre> API</a> will return all cars with Fiat brand and Bravo model.</p> Brand and model are so called query parameters</strong>.</p> As you might already notice, to get resource information, we have always used GET</strong> verb</em></p> Update resources</h3> The API</a> should also support updating resources. Like reading resources, to update a resource we have to specify which resource we want to update, so we again need an identifier</em>.</p> Before, we wanted to get information (read) and we used GET</strong> verb</em>. Now the only difference is the verb.</p> We want to update</strong> so we use the equivalency HTTP</a> verb: PUT</strong>.</p> PUT https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f </code></pre> Actually something else is missing, we have to say what thing of the car we want to change, for example, let's imagine we want to change the engine power and set it to 100CV.</p> We have to send the new engine power to following URL</a> https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f</a> through HTTP</a> using PUT</strong> verb.</p> HTTP</a> protocol allows sending data within PUT message, we have to choose a sending format.</p> We can use JSON</a> or XML</a> or whatever, we only have to ensure that sent format is expected in server side.</p> Note:</strong> Designing a REST</a> API</a> requires to select a data format.</p> </blockquote> JSON example:</strong></p> { "enginePower": 100 } </code></pre> Delete Resources</h3> Let's imagine that now we are the car dealer and we don't want to sell the Fiat Bravo Emotion 1.9CV anymore (the cce05bee-386b-11e5-a151-feff819cdc9f). We'll keep the URL</a> that identifies the resource, but we change the verb: we don't want to read (GET), we don't want to update (PUT), we want to delete (DELETE)</strong>.</p> DELETE https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f </code></pre> We don't have to supply any additional info, only the verb (DELETE) and the resource identifier.</p> Create Resources</h3> And the last verb is create (POST)</strong>. In this case we don't have to identify the resource, because it still doesn't exist.</p> POST https://cardealer.com/api/cars/ </code></pre> But we have to send the data to create the resource.</p> Following with the example, let's create a new car, so we include the necessary data within POST HTTP</a> message, it is something similar what we did at section "Update resources", but we are going to send all required data</strong>, not only the engine power.</p> JSON example:</strong></p> { "brand": "Fiat", "model": "Bravo", "year": 2010, "doors": 5, "enginePower": 120, "version": "Emotion", "clima": true, "ac": false, "fuel": "Diesel" } </code></pre> We can delegate on the system to assign a new identifier</strong>, or simply send it within the message:</p> { "identifier": "cce05bee-386b-11e5-a151-feff819cdc9f", "brand": "Fiat", "model": "Bravo", "year": 2010, "doors": 5, "enginePower": 120, "version": "Emotion", "clima": true, "ac": false, "fuel": "Diesel" } </code></pre> Collections</h3> All actions we have already explained were actually applied over a cars collection.</p> But, what happens if a resource has a nested collection?</p> Continuing with cars example, a car can use a set of engine oils. So the API</a> must allow update, delete or create elements in the set.</p> Note:</strong> For the example we will assume that the oil identifier</em> is the attribute type</em>.</p> </blockquote> Add an element to collection</h4> When we add a car to cars collection, what we do is create a new car, so it is the case of "Create Resources".</p> To add a new engine oil to the car cce05bee-386b-11e5-a151-feff819cdc9f, that already exists:</p> Request:</strong></p> POST https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/ </code></pre> Response:</strong></p> { "type": "5W30", "otherInfo": "This is the best oil for this car" } </code></pre> If we want to add another one:</p> Request:</strong></p> POST https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/ </code></pre> Response:</strong></p> { "type": "10W30", "otherInfo": "This is very good for cold weather" } </code></pre> Update a collection item</h4> If we want to update the info of oil 5W30</em> of car cce05bee-386b-11e5-a151-feff819cdc9f</em>:</p> Request:</strong></p> PUT https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/5W30/ </code></pre> Response:</strong></p> { "type": "5W30", "otherInfo": "This is no longer the best oil for this car" } </code></pre> Delete a collection item</h4> To delete an oil 10W30</em> from car cce05bee-386b-11e5-a151-feff819cdc9f</em>:</p> Request:</strong></p> DELETE https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/10W30 </code></pre> Read a collection item</h4> To get the oil info 10W30</em> of the car cce05bee-386b-11e5-a151-feff819cdc9f</em>:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/10W30 </code></pre> List collection items</h4> As we have seen at "Read a collection item", we can get the info of every collection element, but we also can get multiple collection elements, sorted, paged and apply typical collection actions.</p> We can get all supported oils for a car cce05bee-386b-11e5-a151-feff819cdc9f</em>, it is as simple as:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/ </code></pre> We can also get sorted items:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/?sort_by=type&order=asc </code></pre> We can ask API</a> to return the first 10 oils for car cce05bee-386b-11e5-a151-feff819cdc9f</em>:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/?number_of_elements=10 </code></pre> API</a> can support also pagination:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/?page=3&number_of_elements=2 </code></pre> Above request is telling API</a> that returns the page 3 of all oils of car cce05bee-386b-11e5-a151-feff819cdc9f</em> and it has to show 2 oils per page. If we want to go to next page:</p> Request:</strong></p> GET https://cardealer.com/api/cars/cce05bee-386b-11e5-a151-feff819cdc9f/oils/?page=4&number_of_elements=2 </code></pre> All those features are supported by query parameters</strong>.</p> Common mistake</h2> First time I tried to design a API</a> REST</a> I designed an API</a>, but not REST</a>.</p> My main mistake was the URL</a>s design, I added my own verbs</em> skipping HTTP</a> verbs</em>.</p> Wrong:</strong></p> POST https://example.com/api/cars/ford-focus/delete-oil/5W30 </code></pre> The correct way:</strong></p> DELETE https://example.com/api/cars/ford-focus/oils/5W30 </code></pre> Build C++ project with Gradle 2014-09-27T00:00:00+00:00 Introduction</h2> I am more and more worried about building, dependency management and distribution of my projects. I'd like to find a tool that unifies those processes with independence of the language. I know several tools which almost fit to what I'm looking for, like SCons</a>, Autotools</a>, Ant</a>, Maven</a> and lately Gradle</a>.</p> I've made several projects with Gradle, but always they were Java</a> and Android</a> projects. In Java projects I've found a Maven replacement, because it is faster, easier and less verbose</del>. Update 2020</strong>: With the experience, now if I have to pick a build system for a Java project I'd pick Maven, especially for a professional project, but it is not the topic of this post.</p> About Android projects I suffered the early adoption of Android Studio + Gradle</a>, although currently I think they are more mature and they work fine.</p> First of all, I have to say: building C/C++/Objective-C projects with Gradle is in incubation</a> phase, although now we can perform advanced tasks like:</p> Generation several artifacts within same project (libraries and executables).</li> Dependency management between artifacts (no versions).</li> Different "flavors" of the same software, e.g: we can generate a "Community" release and other one with more enabled features called "Enterprise".</li> It allows multi-platform binary generation.</li> </ul> As I said, this plugin is still having limitations although they are working on it: Gradle C++ roadmap</a>. If they achieve it I'll leave Autotools (I'm going to regret saying that)</del>. Update 2020</strong>: Actually few years later I am not using Autotools, neither Gradle, but I am using Meson</a> and considering Bazel</a>.</p> Case study</h2> I've extracted all the case study from Gradle user guide for native software</a>. I've adapted the project to be multi-platform with 2 versions "Community" and "Enterprise".</p> The application consists of an executable and a dynamic library. The executable will use the library.</p> Gradle also is able to generate a distributable version and a debug version.</p> You can fork the code on GitHub</a>.</p> Project Structure</h3> We can create whichever directory structure, but it is easier using the proposed by Gradle, if not we'll have to specify where the code is located.</p> This is the project structure:</p> gradle-cpp: Root directory. build.gradle: File where is configured Gradle project, it is the equivalent to: build.xml for Ant, Makefile</a> for C/C++ or pom.xml for Maven.</li> src: Folder where the source code is located. hello: This folder contains the module hello. This module will generate hello library. cpp: This folder contains C++ source files. Hello.cpp: File with the implementation of Hello class.</li> </ul> </li> headers: Folder with header files. Hello.h: Class Hello declaration.</li> Msg.h: File with constants declarations. </li> </ul> </li> </ul> </li> main: This folder contains the module which produces the executable that uses hello library. cpp: This folder contains C++ source files. main.cpp: Source code of main function. </li> </ul> </li> </ul> </li> </ul> </li> build: Folder created automatically by Gradle where it leaves all execution results like unit tests reports, compiled files, package distributions, etc.</li> </ul> </li> </ul> C++ Application</h3> It consists of an executable that uses the functionality implemented at hello</code> library.</p> main.cpp</h4> #include "Hello.h" int main(int argc, char ** argv) { Hello hello ("Pepito"); hello.sayHello(10); return 0; } </code></pre> hello</code> library allows greet n</code>^{1</a></sup> times to someone who is passed as argument to constructor class.</p>} ^{1</sup> 'n' Positive integer</p> </div>}Hello.h</h4> class Hello { private: const char * who; public: Hello(const char * who); void sayHello(unsigned n = 1); }; </code></pre> Building with Gradle</h3> Base case</h4> The only we need to build the application with Gradle is: having Gradle2</a></sup> and the file build.gradle</code>.</p> ^2</sup>Actually Gradle is not required if we use the "wrapper", but we aren't going to explain it here, here you can get more info about Gradle Wrapper</a>.</p> </div> build.gradle (Base Case)</h4> apply plugin: 'cpp' model { components { hello(NativeLibrarySpec) {} main(NativeExecutableSpec) { binaries.all { lib library: "hello" } } } } </code></pre> With this simple file, we'll be able to compile and install the application in Debug mode for the platform where we are executing Gradle (in my case X64).</p> If we execute gradle task</code> from the root of the project, we'll get all the tasks we can do with Gradle.</p> In our case, we just want our compiled application ready to run, so we have to execute: gradle installMainExecutable</code>.</p> Once execution has finished, we can run the program calling to build/install/mainExecutable/main</code>3</a></sup>.</p> ^{3</sup> .bat in Windows. Without extension in Linux.</p> </div>}Output</h4> $ build/install/mainExecutable/main 1. Hello Mr. Pepito (Community) 2. Hello Mr. Pepito (Community) 3. Hello Mr. Pepito (Community) 4. Hello Mr. Pepito (Community) 5. Hello Mr. Pepito (Community) 6. Hello Mr. Pepito (Community) 7. Hello Mr. Pepito (Community) 8. Hello Mr. Pepito (Community) 9. Hello Mr. Pepito (Community) 10. Hello Mr. Pepito (Community) </code></pre> Different "Flavors"</h4> With a few lines more we can generate different versions of same application. In our example we are going to build "Community" and "Enterprise" flavors.</p> build.gradle (Flavors)</h4> apply plugin: 'cpp' model { flavors { community enterprise } components { hello(NativeLibrarySpec) { binaries.all { if (flavor == flavors.enterprise) { cppCompiler.define "ENTERPRISE" } } } main(NativeExecutableSpec) { binaries.all { lib library: "hello" } } } } </code></pre> We also have to prepare our application to use compilation parameters.</p> Msg.h</h4> #ifdef ENTERPRISE static const char * EDITION = "Enterprise"; #else static const char * EDITION = "Community"; #endif </code></pre> In this way it selects a string depending on used flavor.</p> If we execute gradle clean task</code> in the root folder, we'll get more available tasks. Before, we had installMainExecutable</code> which has been replaced by installCommunityMainExecutable</code> and installEnterpriseMainExecutable</code>.</p> If we execute both tasks, we'll get the application installed in both flavors:</p> $gradle installEnterpriseMainExecutable installCommunityMainExecutable :compileEnterpriseHelloSharedLibraryHelloCpp :linkEnterpriseHelloSharedLibrary :enterpriseHelloSharedLibrary :compileEnterpriseMainExecutableMainCpp :linkEnterpriseMainExecutable :enterpriseMainExecutable :installEnterpriseMainExecutable :compileCommunityHelloSharedLibraryHelloCpp :linkCommunityHelloSharedLibrary :communityHelloSharedLibrary :compileCommunityMainExecutableMainCpp :linkCommunityMainExecutable :communityMainExecutable :installCommunityMainExecutable BUILD SUCCESSFUL Total time: 9.414 secs </code></pre> Now we can run the application in both flavors:</p> Community</h5> $ build/install/mainExecutable/community/main 1. Hello Mr. Pepito (Community) 2. Hello Mr. Pepito (Community) 3. Hello Mr. Pepito (Community) 4. Hello Mr. Pepito (Community) 5. Hello Mr. Pepito (Community) 6. Hello Mr. Pepito (Community) 7. Hello Mr. Pepito (Community) 8. Hello Mr. Pepito (Community) 9. Hello Mr. Pepito (Community) 10. Hello Mr. Pepito (Community) </code></pre> Enterprise</h5> $ build/install/mainExecutable/enterprise/main 1. Hello Mr. Pepito (Enterprise) 2. Hello Mr. Pepito (Enterprise) 3. Hello Mr. Pepito (Enterprise) 4. Hello Mr. Pepito (Enterprise) 5. Hello Mr. Pepito (Enterprise) 6. Hello Mr. Pepito (Enterprise) 7. Hello Mr. Pepito (Enterprise) 8. Hello Mr. Pepito (Enterprise) 9. Hello Mr. Pepito (Enterprise) 10. Hello Mr. Pepito (Enterprise) </code></pre> Release or Debug</h4> By default Gradle compiles in Debug mode, but we can add the Release mode which enables several optimizations and remove debug flags^{4</a></sup>.</p>} ^{4</sup> We can also specify/modify which optimizations to apply.</p> </div>}build.gradle (Release/Debug)</h4> apply plugin: 'cpp' model { buildTypes { debug release } // ... the rest of file below doesn't change } </code></pre> If we execute gradle clean task</code> we will get more tasks, they have been split, for example installCommunityMainExecutable</code> has been split in installDebugCommunityMainExecutable</code> and installReleaseCommunityMainExecutable</code>.</p> Multi-platform</h4> Also we can use cross-compiling features provided by compilers and generate native components for other platforms. To do so we just have to add the supported platforms.</p> This only works if we have installed the Toolchain</a> for the target platform.</p> build.gradle</h4> apply plugin: 'cpp' model { buildTypes { debug release } platforms { x86 { architecture "x86" } x64 { architecture "x86_64" } itanium { architecture "ia-64" } } flavors { community enterprise } components { hello(NativeLibrarySpec) { binaries.all { if (flavor == flavors.enterprise) { cppCompiler.define "ENTERPRISE" } } } main(NativeExecutableSpec) { binaries.all { lib library: "hello" } } } } </code></pre> When execute gradle clean task</code> we get the different build options we have.</p> Note:</strong> In this example, we can build different versions of the application in different flavors for different platforms in Debug or Release mode.</p> </blockquote> Try it yourself</h3> You can find the project at https://github.com/carlosvin/cpp_gradle</a>.</p> Requirements:</p> Java 6 or higher.</li> An installed compiler (e.g GCC</a>)</li> </ul> You just have to follow next steps:</p> git clone git@github.com:carlosvin/cpp_gradle.git</code></li> cd cpp_gradle</code></li> ./gradlew task</code> or ./gradlew.bat task</code> if you are in Windows. In this way you'll see available tasks for this project. The first execution will take more time, because it downloads Gradle runtime.</li> If you are in a 64 bits platform, you can use this command to install the application: ./gradlew installX64ReleaseEnterpriseMainExecutable</code>.</li> Run the application you just built: build/install/mainExecutable/x64ReleaseEnterprise/main</code>.</li> </ol> Conclusions</h2> With a small configuration file, we can achieve many different build combinations.</p> Gradle for C++ has a promising future, and I hope it will follow the successful path of Java and Android support.</p> It is well supported by continuous integration systems and offers many plugins and features.</p> However, Gradle for C++ is still under development, so we need to be cautious:</p> Do not use it in production environments.</li> Many things may change or be removed in future versions.</li> </ul> The full example is available at https://github.com/carlosvin/cpp_gradle</a>. I encourage you to try it yourself.</p> Note:</strong> If you find any issues in this example, please leave a comment, open an issue, or submit a fix at https://github.com/carlosvin/cpp_gradle</a>.</p> </blockquote> For more information, see Getting Started with Gradle Native</a>.</p> Embedded Java Databases - Performance Comparison 2014-06-07T00:00:00+00:00 Embedded Databases</h2> These are databases that do not require a server, are embedded within the application itself, and are usually stored in local files. This, combined with the fact that they often have a mode where data is kept in memory, can result in very high performance.</p> However, this high degree of coupling to the application means they perform worse when shared between multiple applications due to access collisions.</p> Another advantage is that you don't have to maintain and manage a database server.</p> I'm going to do a performance comparison between 3 transactional embedded databases (ACID). NoSQL databases are not included in this comparison as they are in a different performance league.</p> Contents</p> SqliteDB</h3> This is a library written in ANSI C</a>, less than 500KB, multi-platform, with no external dependencies, and stores all database content in a single file.</p> It gives the best performance in the test results below.</p> It can be used from C and C++, but also from other programming languages</a> (PHP, Python, Java, .NET ...).</p> In the case of Java, we can manage this database through JDBC. The library can be obtained from here</a>.</p> Here is an example of inserting a series of objects of the class Price.java</a>:</p> public class JdbcDb implements DB { enum Column { INSTRUMENT, MARKET, PRICE, DATE; } public static final String TABLE_NAME = "PRICES"; public static final String SQL_INSERT = "INSERT INTO " + TABLE_NAME + " (INSTRUMENT, MARKET, PRICE, DATE) VALUES (?,?,?,?)"; // ... private void insert(Price p, PreparedStatement preparedStatement) throws SQLException { preparedStatement.setString(1, p.getInstrument()); preparedStatement.setString(2, p.getMarket()); preparedStatement.setDouble(3, p.getPrice()); preparedStatement.setTimestamp(4, new Timestamp(p.getDate().getTime())); preparedStatement.addBatch(); } @Override public void insert(Price... prices) throws SQLException { PreparedStatement preparedStatement = conn.prepareStatement(SQL_INSERT); for (Price p : prices) { insert(p, preparedStatement); } preparedStatement.executeBatch(); preparedStatement.close(); conn.commit(); } // ... } </code></pre> DerbyDB or JavaDB</h3> JavaDB is an Oracle distribution of the open-source DerbyDB database. It supports the ANSI/ISO SQL</a> standard via JDBC and Java EE</a>. These libraries are included in the JDK.</p> It stores the database in multiple files, which can be useful for scaling storage.</p> It can only be used in Java, not from other languages.</p> As we will see later, it was the slowest in the test results.</p> The example implementation for inserting a series of objects of the class Price.java</a> is exactly the same as above for SqliteDB. This is one of the benefits of JDBC, which allows us to obtain a connection for a specific database, but from there we can almost always forget about the specific database as long as it supports our SQL queries.</p> ObjectDB</h3> This is an object-oriented database</a> that allows access via JPA</a>, a Java standard that aims to retain the advantages of object orientation, which are often lost when dealing with persistence layers.</p> It is really easy to perform typical database actions, abstracting away from SQL. For example, here is how you would insert an array of Price objects with ObjectDB:</p> public void insert(Price... prices) throws SQLException { em.getTransaction().begin(); for (Price p : prices) { em.persist(p); } em.getTransaction().commit(); } </code></pre> If you add this ease of use to the fact that its performance in the test results was very good (close to SqliteDB), I can say that I have discovered an embedded database to consider for future Java projects. However, as with JavaDB, it is only useful if you are programming in Java.</p> The Tests</h2> The tests simply consist of repeatedly performing a series of CRUD</a> operations. For each database, the test performs a series of inserts, selects, updates, and deletions of objects of the Price class.</p> I used 100,000 instances of the Price.java</a> class, the same for each type of database.</p> For this, I created a DB.java</a> interface that each implementation for each database shares.</p> public interface DB { public void insert(Price... prices) throws SQLException; public void createTable(); public void deleteAll(); public void update(Price... prices) throws SQLException; public Set<Price> selectAll(); } </code></pre> JDBC Databases</h3> Only the data needed to obtain the driver and the database connection changes, so all the logic is in the JdbcDb.java</a> class, from which SqliteDB and DerbyDB inherit.</p> Source Code</h3> You can download the source code from here</a>.</p> It is a Gradle project, so to run the tests you just have to execute: gradle test</code>.</p> Gradle will automatically download the necessary libraries, compile, and run the tests.</p> You can also see the execution directly at Travis CI</a>.</p> </p> Test Results</h3> db.DbTest > testSqliteDB STANDARD_OUT Testing with 100000 elements 100000 Prices SqliteDB, total time 3455 ms <1> Insert: 1508 ms Select: 605 ms Update: 1319 ms Delete: 23 ms ---------------------------------------------- db.DbTest > testObjectDB STANDARD_OUT Testing with 100000 elements 100000 Prices ObjectDB, total time 6467 ms <2> Insert: 2579 ms Select: 1126 ms Update: 1698 ms Delete: 1064 ms ---------------------------------------------- db.DbTest > testDerbyDB STANDARD_OUT Testing with 100000 elements 100000 Prices DerbyDB, total time 24808 ms <3> Insert: 11467 ms Select: 695 ms Update: 6983 ms Delete: 5663 ms </code></pre> <1> SqliteDB is the fastest. <2> ObjectDB is twice as slow as SqliteDB. <3> JavaDB or DerbyDB is the slowest, about 8 times slower than SqliteDB.</p> Java serialization ways: Performance Comparison 2014-05-13T00:00:00+00:00 Recently I've had to serialize/deserialize some data in Java</a> binary format. Lately I use JSON</a> or XML</a> formats.</p> I remember that to serialize Java</a> objects they must implement the Serializable</a> interface, but I had also read in Internet other way, implementing the Externalizable</a> interface, then, which interface must I implement? It depends on what you want such as everything in the life.</p> When to use Serializable</a> or Externalizable</a>.</p> Serializable</h2> To serialize easily. You have to write less code.</li> This way has some restrictions: The object to serialize must implement the default constructor (0 args). It must be responsible to manage the parent class attributes.</li> The performance is not as important, we will see more about that in Performance tests (Serializable vs. Externalizable)</a>.</li> </ul> Externalizable</h2> You must implement the serialization/deserialization methods, so you have to write more code.</li> When you cannot use Serializable</a>.</li> When you want to improve the performance, as we'll see in Performance tests (Serializable vs. Externalizable)</a>.</li> If you have to manage the serialization of parent class attributes, then I recommend you use Externalizable</a>, because we'll avoid a weird private methods overriding.</li> </ul> private void writeObject(ObjectOutputStream oos) private void readObject(ObjectInputStream ois) </code></pre> Performance tests (Serializable vs. Externalizable)</h2> Serializable</a>: Java, through introspection, guesses the types of class attributes to know how to serialize/deserialize them, but this "magic" is not free, it has a performance penalty.</p> When we use Externalizable</a> interface, we decide how to serialize/deserialize, namely we have to write the code that does it. We've lost ease, but also we avoid that Java</a> does some tasks, so if we override the methods properly, we'll get a performance improvement.</p> To know how big is the performance difference between both interfaces, I've written a tiny example in which we serialize an object with 2 collections with 100000 elements each one</a>. Here you can find the tests execution results</a>. There are 3 different implementations:</p> Implementing Serializable</h3> As we mentioned above, Java</a> has to guess certain things. During this process it sacrifices some of performance (slowest way), in exchange we get really simple source code, the class to serialize just has to implement the Serializable</a> interface.</p> public class Contacts implements Serializable { // ... } </code></pre> serializing: 1133 millisecond</li> deserializing: 506 millisecond</li> </ul> Implementing Externalizable (wrong way)</h3> If the class implements Externalizable</a>, we must tell to Java</a> how it has to serialize/deserialize the class attributes. We just have to be careful, because if we did it bad, then we'll get the worst of the both worlds: more complex implementation and bad performance, i.e: If we serialize/deserialize complex class attributes (like collections), Java</a> will also have to guess many things about the attributes type.</p> @Override public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException { setEmails((Set<String>) in.readObject()); setPhones((Set<String>) in.readObject()); } @Override public void writeExternal(ObjectOutput out) throws IOException { out.writeObject(emails); out.writeObject(phones); } </code></pre> serializing: 737 millisecond</li> deserializing: 367 millisecond</li> </ul> Implementing Externalizable (right way)</h3> If we serialize one by one the collection elements, then we'll save more time, because Java</a> serializes simple types, this way avoids guessing things that we actually know.</p> @Override public void readExternal(ObjectInput in) throws IOException, ClassNotFoundException { emails.clear(); phones.clear(); int nEmails = in.readInt(); for (int i = 0; i < nEmails; i++) { emails.add(in.readUTF()); } int nPhones = in.readInt(); for (int i = 0; i < nPhones; i++) { phones.add(in.readUTF()); } } @Override public void writeExternal(ObjectOutput out) throws IOException { out.writeInt(emails.size()); for (String e : emails) { out.writeUTF(e); } out.writeInt(phones.size()); for (String p : phones) { out.writeUTF(p); } } </code></pre> serializing: 204 millisecond</li> deserializing: 92 millisecond</li> </ul> We've gained performance at expense of write more code.</p> Results Analysis</h3> We don't gain performance due to use an interface or the other one.</p> We gain performance because Externalizable</a> interface forces us to implement ourselves the guessing code, so Java</a> doesn't have to do that.</p> As we have seen at Implementing Externalizable (wrong way)</a>, si no tenemos cuidado, conseguiremos una mejora poco significativa a costa de complicar nuestro código fuente.</p> Test results</a>.</li> Code in Github</a>.</li> </ul> Mutex C++ 2014-04-24T00:00:00+00:00 C++11</a> has added many improvements to help us developing multi-thread systems. I'm going to talk about Mutex</a>.</p> In previous C++11 compiler versions</a>, we can get a pthread</a> mutex, but we must initialize it and destroy it in old C style, in the end you must do more things than just lock/unlock.</p> With C++11 Mutex Class</a>, we just lock/unlock the object.</p> #include <mutex> std::mutex mtx; void do_something (int i) { mtx.lock(); // critical section mtx.unlock(); } </code></pre> Or you can just use Mutex with a generic lock guard</a>.</p> Tip:</strong> If you still have to stick to previous C++11 compiler versions, maybe it is useful to you a wrapper class I created that helps you to work with pthread mutex, so you just have to lock/unlock the Mutex object: Gist code</a>.</p> </blockquote> Android Studio Portable Download 2013-10-29T00:00:00+00:00 If you want to use Android Studio</a> without installation, as a portable software, here you have the link to last portable version of Android Studio</a>.</p> Debugging library generated with libtool 2013-02-01T15:30:00+00:00 When debugging an executable that uses a library generated with libtool</a>1</a></sup>, you might encounter the following error:</p> $ gdb ./tests-mylib "tests-mylib": not in executable format: File format not recognized </code></pre> libmylib.so</code>: is a dynamic library generated with libtool.</li> tests-mylib</code>: is an executable that uses the mylib library.</li> </ul> For those thinking I forgot to add the -g</code> compilation option, this error occurs even when using -g</code>.</p> The error is due to the fact that we are trying to execute a script generated by libtool, which is a wrapper over the actual program to facilitate its execution2</a></sup>.</p> To debug our test program tests-mylib</code></h2> libtool --mode=execute gdb tests-mylib </code></pre> ^1</sup>Tool that belongs to the Autotools</a> used to create portable software libraries.</p> </div> ^2</sup>More information in libtool documentation</a>.</p> </div> C++ Struct memory alignment 2012-11-26T00:00:00+00:00 A C++ struct is an element that groups attributes with different types so we can manipulate them all together using same reference. It is like a class with public visibility by default for functions and attributes.</p> If we want to work in a lower level, closer to machine, it might be useful to understand how that data structure is stored in memory and how to control that mapping.</p> Struct example</h2> It has two attributes: an integer (4 bytes) and a boolean (1 byte).</p> struct SampleStruct { bool flag; unsigned int timeout; }; </code></pre> If we get the instance size using sizeof</code> we should get 5 bytes size and memory would be like:</p> </p> But</strong> is not that simple, memory alignment depends on compiler and system. We will learn how to control compiler alignment policy, so we can avoid getting unexpected allocation memory sizes.</p> For example, in my local host, if I get the sizeof</code> the previous structure without pragma</code> declarations, I get a 8 bytes size</strong>. We are getting 8 Bytes instead of expected 5 Bytes because the compiler allocates more memory at the end of structure so it fits in 2n bytes blocks. Memory actually looks like:</p> </p> Example output:</p> SampleStruct flag: 1 Bytes + timeout: 4 Bytes = 5 Bytes sizeof struct: 8 Bytes -- </code></pre> TIP: If we want to know the exact structure size we have to specify compiler how to align the memory, to do so we have #pragma pack(n)</code> directive.</p> #pragma pack</code> directive in C++ struct</h2> It is a preprocessor directive to indicate to compiler how to align data in memory.</p> Example with different memory alignment configurations:</p> #pragma pack (1) struct SampleStructPack1 { bool flag; unsigned int timeout; }; #pragma pack(0) #pragma pack (2) struct SampleStructPack2 { bool flag; unsigned int timeout; }; #pragma pack(0) #pragma pack (4) struct SampleStructPack4 { bool flag; unsigned int timeout; }; #pragma pack(0) struct SampleStruct { bool flag; unsigned int timeout; }; </code></pre> Example output:</p> SampleStructPack1 flag: 1 Bytes + timeout: 4Bytes = 5Bytes sizeof struct: 5 Bytes -- SampleStructPack2 flag: 1 Bytes + timeout: 4Bytes = 5Bytes sizeof struct: 6 Bytes SampleStructPack4 flag: 1 Bytes + timeout: 4Bytes = 5Bytes sizeof struct: 8 Bytes SampleStruct flag: 1 Bytes + timeout: 4Bytes = 5Bytes sizeof struct: 8 Bytes </code></pre> SampleStructPack1 #pragma pack (1)</code>: It allocates 1 byte memory block, so our sample struct fits perfectly, in this case it is true that 4 + 1 = 5</code>.</li> SampleStructPack2 #pragma pack (2)</code>: Minimum block size is 2 bytes. Integer attribute fits because it just needs 2 blocks of 2 Bytes. Boolean attribute needs just 1 Byte, but minimum block size is 2 Bytes, that's why total allocated memory is 6 bytes, 4 + 2 = 6</code>.</li> SampleStructPack4 #pragma pack (4)</code>: It is like previous one, but in this case we are wasting more memory for boolean attribute, it needs 1 Byte, but we are allocating 4 Bytes.</li> SampleStruct (default compiler alignment): As you can see it behaves exactly like #pragma pack (4)</code>, so we can deduce it is the default compiler alignment.</li> </ol> IMPORTANT:</strong> Why don't we always use smallest memory alignment (#pragma pack (1)</code>) so we can save more memory?</p> WARNING:</strong> Because of performance loss.</p> Performance test</h2> The test will allocate same number of elements in arrays for each structure type (1, 2, 4).</p> Example output:</p> SampleStructPack1: 500000000000000000 bytes allocated in 94311 nanoseconds SampleStructPack2: 600000000000000000 bytes allocated in 1777 nanoseconds SampleStructPack4: 800000000000000000 bytes allocated in 1519 nanoseconds </code></pre> As you can see, the smallest memory alignment spends more time allocating and releasing memory.</p> Performance test source code:</p> #include <iostream> #include <chrono> #pragma pack (1) struct SampleStructPack1 { bool flag; unsigned int timeout; }; #pragma pack(0) #pragma pack (2) struct SampleStructPack2 { bool flag; unsigned int timeout; }; #pragma pack(0) #pragma pack (4) struct SampleStructPack4 { bool flag; unsigned int timeout; }; #pragma pack(0) struct SampleStruct { bool flag; unsigned int timeout; }; static const long MAX_ELEMENTS = 100000000000000000; using namespace std; using namespace std::chrono; void allocate1() { SampleStructPack1 elements [MAX_ELEMENTS]; cout << "SampleStructPack1: " << sizeof(elements) << " bytes allocated"; } void allocate2() { SampleStructPack2 elements [MAX_ELEMENTS]; cout << "SampleStructPack2: " << sizeof(elements) << " bytes allocated"; } void allocate4() { SampleStructPack4 elements [MAX_ELEMENTS]; cout << "SampleStructPack4: " << sizeof(elements) << " bytes allocated"; } void chrono1() { auto begin = high_resolution_clock::now() ; allocate1(); cout << " in " << duration_cast<nanoseconds>(high_resolution_clock::now() - begin).count() << " nanoseconds" << endl; } void chrono2() { auto begin = high_resolution_clock::now() ; allocate2(); cout << " in " << duration_cast<nanoseconds>(high_resolution_clock::now() - begin).count() << " nanoseconds" << endl; } void chrono4() { auto begin = high_resolution_clock::now() ; allocate4(); cout << " in " << duration_cast<nanoseconds>(high_resolution_clock::now() - begin).count() << " nanoseconds" << endl; } int main(int argc, char *argv[]) { chrono1(); chrono2(); chrono4(); return 0; } </code></pre> Git Proxy Configuration 2012-11-15T12:00:00+00:00 Working with Git through a corporate proxy can be challenging. Here's a quick guide to configure Git proxy settings.</p> Configure HTTP Proxy</h2> git config --global http.proxy http://<username>:<password>@<ip_host>:<port> </code></pre> Configure HTTPS Proxy</h2> git config --global https.proxy https://<username>:<password>@<ip_host>:<port> </code></pre> Disable Proxy</h2> git config --global --unset http.proxy git config --global --unset https.proxy </code></pre> Parameters</h2> username</code>: Proxy username</li> password</code>: Proxy password</li> ip_host</code>: Proxy server address</li> port</code>: Proxy server port</li> </ul> Verify Configuration</h2> git config --global --get http.proxy git config --global --get https.proxy </code></pre> More secure: Environment Variables</h2> Using environment variables is a more secure method, as entering credentials directly in commands can expose them in your shell history. Instead, set your proxy credentials using environment variables:</p> export HTTP_PROXY=http://<username>:<password>@<ip_host>:<port> export HTTPS_PROXY=https://<username>:<password>@<ip_host>:<port> </code></pre> Note: Setting environment variables in the shell will also save them in the history, so this approach is more secure if you just set the environment variables in a file like ~/.bash_profile</code> (if you are using bash shell</a>).</p> For more details, see the official Git configuration documentation</a>.</p> Security Note:</strong> Storing credentials in environment variables or in files like ~/.bash_profile</code> is not inherently secure, as these files are stored in plain text and can be read by anyone with access to your home directory. For better security, consider using Git credential helpers</a> or a dedicated credential manager. If you must store sensitive data in profile files, restrict access with chmod 600 ~/.bash_profile</code> to limit who can read the file.</p> Replace punctuation symbols in Python 2012-10-23T00:00:00+00:00 Below is an explanation of how to replace punctuation symbols with whitespace in Python.</p> import re, string def remove_punctuation(text): return re.sub('[%s]' % re.escape(string.punctuation), ' ', text) </code></pre> Calling the previous function:</strong></p> >>> remove_punctuation("El perro, de San Roque, no tiene rabo; ni nunca lo ha tenido.") </code></pre> We will get this output:</strong></p> 'El perro de San Roque no tiene rabo ni nunca lo ha tenido ' </code></pre> We could make the function a little bit more generic to replace punctuation symbols with any other string.</p> import re, string def replace_punctuation(text, replace): return re.sub('[%s]' % re.escape(string.punctuation), replace, text) </code></pre> Calling the function to replace punctuation symbols by "[stop]":</strong></p> >>> replace_punctuation( "El perro, de San Roque, no tiene rabo; ni nunca lo ha tenido.", '[stop]') # output 'El perro[stop] de San Roque[stop] no tiene rabo[stop] ni nunca lo ha tenido[stop]' </code></pre> Text Normalization in Python 2012-10-02T18:00:00+00:00 In many languages, such as Spanish, there are characters that do not have ASCII</a> representation, such as á</strong>, which does have representation in Unicode</a>.</p> To avoid problems or for simplification, an equivalence has been established between Unicode</a> and ASCII</a> characters. Below I'll show you a piece of Python</a> code that performs this conversion.</p> # -*- coding: utf-8 -*- from unicodedata import normalize def normalize_text(text): return normalize('NFKD', text) # (1) .encode('ASCII', 'ignore') # (2) </code></pre> We specify the normal form that we apply in the normalization. In this case NFKD</code>. More information about normal form types</a>.</li> We convert the normalization result to ASCII. In case a character is erroneous, it will simply be ignored.</li> </ol> Running the function</h2> >>> normalize_text('aáaá eéeé iíií oóoó ñnñn AÀAÀ') b'aaaa eeee iiii oooo nnnn AAAA' </code></pre> Remove HTML Tags 2012-10-02T11:30:00+00:00 Below we'll see how to remove HTML tags from a character string.</p> With Python</h2> Function responsible for removing HTML tags:</p> main.py</h3> import re def strip_tags(value): return re.sub(r'<[^>]*?>', '', value) </code></pre> Let's test an HTML fragment with the strip_tags</code> function:</p> Example usage</h3> import re def strip_tags(value): return re.sub(r'<[^>]*?>', '', value) html_text = """ <!DOCTYPE HTML> <html> <head> <title>Title</title> </head> <body> <p>Paragraph</p> </body> </html>""" print(strip_tags(html_text)) </code></pre> If we run the script we get the following result:</p> Title Paragraph </code></pre> If it was easy with Python</a>, let's see how it works with Django</a>.</p> Django</h2> Django</a> offers a function for this: strip_tags</code></a>.</p> First, you just need to install the library: pip install django</code>.</p> Using Django's strip_tags</h3> from django.utils.html import strip_tags html_text = """ <!DOCTYPE HTML> <html> <head> <title>Title</title> </head> <body> <p>Paragraph</p> </body> </html>""" print(strip_tags(html_text)) </code></pre> Remove orphaned packages (Linux) 2012-10-02T00:00:00+00:00 When we install a package in most Linux distributions, the package system installs other packages needed by the package that we are installing. If we uninstall the package, its dependencies might not be uninstalled; these unused dependencies are called orphaned packages.</p> Let's see how to remove orphaned packages.</p> Archlinux</h2> pacman -Rsn $(pacman -Qdtq) </code></pre> How does the command work?</p> pacman -Qdt</code> lists all orphaned packages</li> pacman -Rsn</code> uninstalls the listed packages</li> </ul> Debian</h2> apt-get remove --purge $(deborphan) </code></pre>

My software engineering journey

Building AI-Promptable Full-Stack Apps: A Reproducible Architecture

Isolated Integration Tests with pytest-scenarios

The Problem</h2> Consider testing an e-commerce checkout system. You need to verify that:</p> Active customers can complete orders</li> Suspended customers are rejected</li>

The Solution: Declarative Test Scenarios</h2> With pytest-scenarios</code>, you declare what state you need</em>, and the plugin handles the rest:</p>

Key Concepts</h2>

Templates Define Defaults</h3> Define templates with sensible defaults—tests only override what matters:</p>

Automatic Cleanup</h3> The plugin clears all managed collections before each test</em>. Even if a test fails, the next test starts fresh.</p>

About

TanStack Router: Opinionated Guidelines for Production React Apps

Typescript</h3> Faster and safer development: Our goal was to catch bugs as early as possible—ideally at compile time, together with a next level developer experience. This approach not only reduces runtime errors but also makes refactoring and onboarding significantly easier.</p>

Project guidelines</h2>

Exceptions in loaders</h4> Throw in loaders (e.g., AppError</code>) so route error components handle failures consistently. Centralize error typing so UIs can show banners/toasts with title</code> and description</code>.</p>

Leverage the default router cache</h3> The built‑in cache works well by default. After a mutation followed by navigation, you may want to router.invalidate()</code> to refetch fresh data.</p>

Easily creating a golang command line tool

Parameterized Tests in Cypress

Test results</h3> </p>

Generated input data set</h2> Instead of explicitly defining a data set, you can generate it, for example, creating a random data set.</p> Example checking that any generated random number is less than 0:</p>

Test results</h3> </p>

Golang over Java for REST services

Validations</h2>

Java + Spring.io Validation</h3> You can enable the validation in the controller level, then in the handlers you can also specify the type of validation. Let's explain it using the previous example:</p>

Dependency injection / IoC</h2>

Unit tests</h3> For unit tests there are no big differences.</p>

Create a PWA with Create React App: Lessons Learned

Offline first, not really</h4> Important</strong>: This section is not relevant if your use case doesn’t imply saving data linked to the user.</p> </blockquote>

Data model</h4> Firestore API is easy and intuitive, I really like it, but don’t assume it will have same features as other document DBs or SQL DBs.</p>

Create SPA with React + Typescript + Parcel

Step by step project creation</h2> In this section I will describe how I created this project.</p> Firstly, create package.json</code> with yarn init</a> command.</p> yarn init </code></pre> Add required dependencies</h3> Add React dependencies.</p>

Add required dependencies</h3> Add React dependencies.</p>

Create application source code</h3> First we create the React application in src/index.tsx</code> file.</p>

Add commands build the project</h3> I’ve added the commands:</p> build</code>: Check "Build production bundle"</em> section.</li>

Custom Maven Plugin - Override default build lifecycle

Generate the project using archetype</h3> mvn archetype:generate \ -DarchetypeGroupId=com.github.carlosvin.archetype \ -DartifactId=lifecycle-maven-plugin-archetype \ -DarchetypeVersion=0.6 </code></pre>

Example how to create custom Maven Plugin

Write a custom Mojo</h3> It is quite straightforward to implement a Mojo class, we have to:</p>

Automatic Cleanup</h3>
The plugin clears all managed collections before each test</em>. Even if a test fails, the next test starts fresh.</p>

Leverage the default router cache</h3>
The built‑in cache works well by default. After a mutation followed by navigation, you may want to `router.invalidate()</code> to refetch fresh data.</p>`

Test results</h3>
</p>

Test results</h3>
</p>

Tip:</strong> Please check the cypress official examples to learn more about dynamic/parameterized testing</a>.</p> </blockquote>

Java + Spring.io Validation</h3>
You can enable the validation in the controller level, then in the handlers you can also specify the type of validation. Let's explain it using the previous example:</p>

Unit tests</h3>
For unit tests there are no big differences.</p>
Go</a> comes with a standard library for testing and benchmarking</a>.</p>
For Java</a> there are many well-known unit testing frameworks, but Spring already has quite big support for unit testing</a>.</p>

Generate the project using archetype</h3>
`mvn archetype:generate \ -DarchetypeGroupId=com.github.carlosvin.archetype \ -DartifactId=lifecycle-maven-plugin-archetype \ -DarchetypeVersion=0.6 </code></pre>`

Write a custom Mojo</h3>
It is quite straightforward to implement a Mojo class, we have to:</p>