{"componentChunkName":"component---src-templates-blog-post-js","path":"/microservices-node/","result":{"data":{"markdownRemark":{"html":"

GitTix (A Simple StubHub Clone):

\n

Overview:

\n\n

Future Enhancements:

\n\n

DB Resources :

\n\n

In this app we will follow a resource-based microservice design, this basically means each resource will has its own microservice.\nOf course a feature based design could be a better approach, but i am trying to start simple a t first.

\n

Microservices :

\n\n

\n \n \n \n \n \n\n \n \n \n \n

\n

This is a microservices application that uses Async communication between services, so it would be a great idea to list the events that we will work with through out the App.

\n

Events (Services use for Communication):

\n\n

Application Architecture Overview:

\n

We're going to use Next.js for client (SSR), and mongodb for other backend services, except expiration service, we are going to use Redis.

\n

Also, for the event bus server, we are going to use NATS.

\n

\n \n \n \n \n

\n

Kubernetes Cluster Config:

\n\n

Creating Secret inside k8s:

\n
- We run this command `kubectl create secret generic SECRET_NAME --from-literal KEY=VALUE`\n
\n

Password hashing Reminder :

\n\n

Is user Logged In (Microservices Architecture):

\n\n

Sometimes with each each request we could ask for user permission from auth server (if that was a requirement)

\n\n

Server Side Rendering:

\n\n

NATS Streaming Server:

\n\n

Common Concurrency Issues:

\n
    \n
  1. \n

    If a service fails to process event, and other events are processed on other services instances, this means events are processed out of order, which could be unacceptable depending on the business case.

    \n
    2. \n
  2. \n

    Maybe one listener(service) runs more quickly than another. Causing some events being processed out of order.

    \n
    3. \n
  3. \n

    Depending on hearbeat settings (and the service being shut unGracefully), NATS might think a dead service to be alive and send it an event to be processed, causing this event to be delayed.

    \n
    4. \n
  4. \n

    Race condition may occur caused by an event being processed twice. This could occur if one listener is very slow and is about to finish in 29.999s (NATS consider event as failed after 30s and did not recieve ack from service). So after 1ms NATS consider the event as failed and send it to be processed again, while the first service is still (probably finished processing) processing the event.

    \n
    5. \n
\n

Common Questions:

\n
1. Are concurrency issues related only to Microservices Architecture? No. it occurs also in Monolith apps, but it's just more prominent in MicroServices Arch.\n\n2. Why not run one instance of the listener server? Still the same issue (processing out of order) might occur, and we end up with a bottleneck in our application that we are not able to scale.\n\n3. Could we share state between listener services (SRV-1 => cache/Queue <= SRV-2) and check events sequence number? This solution might seem good, but it strictly process events in sequence and this might be a penalty, if we for example have a multitenant system, tenant(B) is independent from tenant(A), and yet we are processing their events sequencially rather than concurrently, also this issue might occur on one-tenant level application, if we are updating two different records for example. [One Update At A Time]\n\n4. Building on the Idea above, Could we check last processed event by the event # and also the resourceId (or tenantId+resourceId) ? This solution is good (having pool of sequence queues based on resources, in other words, queue per resource that is updated by events) (Meaning, if events are updating 2 resources, we could have one queue for each resource). But we have an issue related to NATS, and that is, the repeated numbering of events is not allowed in the same channel, so we might end up creating many channels which introduces an overhead.\n\n5. A possible solution might be built upon the previous idea, but now the publisher will save the event data.\nThe Steps are: VIDEO{285}\n  a. Pub send event to `NATS`.\n  \n  b. `NATS` send event `seqNum` back to `publisher`, and pub saves this event( `seqNum`, `payload`, `lastSeq` (not used yet) ).\n\n  c. `NATS` forward this event to a `listener`, then it gets processed, and in the database of the `listener`, we check the `lastSeq` attribute of the incoming event and see if it matches the `lastEventIDProcessed` for the resource, then process the event, and finally, we save the `seqNum` of the incoming event as `lastEventIDProcessed` for the resource being updated.\n\n  d. When another event is being published for the same resource (identified by event payload that might have resourceId) publisher checks its database for the last event for this resource, and assign this last event's `seqNum` to the new event's `lastSeq` attribute.\n\n  e. Steps a, b, c are repeated.\n\nNOTE: NATS really does not send he seqNum back, so this solution might work with other solution other than NATS. This could be solved by relying on numbering being from publisher itself.\n
\n

A solution:

\n
- Re-design the services again - We did not mention anything about the publisher, and began to design based only on NATS.\n- But if we take a closer look at publisher service, we know that it needs a database to save the requests/resources it recieves. So, we could rely on numbering these requests as they come and use this transactionNumber as our indicator for events sequence.\n- Also, the listener's database still has the `lastEventNumber` but it now corresponds to the transactionNumber in the publisher service. And the logic should check this `lastEventNumber` before any processing.\n\n- So, numbering the transactions or events as they come and make the consumers aware of this numbering, solves the ordering of events problem we had previously.\n
\n

\n \n \n \n \n

\n\n\n\n\n\n\n
Git Repositories approaches in Microservices Arch
\n
    \n
  1. Mono Repository (Appraoch used)
    2. \n
\n

\n \n \n \n \n

\n\n\n\n\n\n\n\n
Digital Ocean Deployment
\n\n

\n \n \n \n \n

\n","frontmatter":{"title":"Building Ticketing System Microservices Notes","featuredImage":null}}},"pageContext":{"slug":"/microservices-node/"}},"staticQueryHashes":["3649515864"]}