Archive | rapid development RSS feed for this section

Dependency Injection pattern in database development

24 Jun

To get yourself familiar with dependency injection
pattern please check Wikipedia link here:
http://en.wikipedia.org/wiki/Dependency_injection

The primary database focus is to persist data and guaranty data integrity… well, at list we can say it about many databases. As a result database development evolved in its own procedural-way, and it is hard to argue against such a way or to change it, because in most cased database designs have tendency to harden, solidify, and resist a change after they get built. Besides, who want a change when over the years masses of applications got built on that-old-table. Just try to change it and wait for something to fail…
Whatever reason you have for a change try the Dependency Injection pattern. This pattern injects the “depended-on” code to the destination code automatically by knowing the requirement of the destination code and altering an expected outcome of the destination code. Let’s take very popular BOO and FOO templates for our example:

CREATE PROCEDURE dbo.sp_BOO_manager
AS
BEGIN
   IF dbo.FOO() < dbo.FOOTheshold()
      EXEC BOO 'Have to do BOO logic';
      ELSE MOO 'Have to do MOO logic, instead';
END

Here is how to introduce dependency injection into our code:

  • Create following two schemas: INTERFACE, PROD_IMPLEMENTATION
CREATE SCHEMA INTERFACE;
CREATE SCHEMA PROD_IMPLEMENTATION;
  • Move actual functions and procedures with business logic to the PROD_IMPLEMENTATION schema, like this one:
CREATE PROCEDURE PROD_IMPLEMENTATION.BOO
@Message VARCHAR(50)
AS
BEGIN
    PRINT 'Very complicated proprietary logic';
END
  • Create synonyms in the INTERFACE schema that will point to the functions and stored procedures in the PROD_IMPLEMENTATION schema. Note that synonyms can have the same names, because they are located in the different schemas than actual business logic. For instance:
CREATE SYNONYM INTERFACE.BOO
   FOR PROD_IMPLEMENTATION.BOO;
CREATE SYNONYM INTERFACE.FOO
   FOR PROD_IMPLEMENTATION.FOO;
CREATE SYNONYM INTERFACE.FOOTheshold
   FOR PROD_IMPLEMENTATION.FOOTheshold;
  • Then change dbo.sp_BOO_manager stored procedure to use synonyms from the INTERFACE schema instead of objects themselves. Here is what you will get:
CREATE PROCEDURE dbo.sp_BOO_manage
AS
BEGIN
   IF INTERFACE.FOO() < INTERFACE.FOOTheshold()
      EXEC INTERFACE.BOO 'Have to do BOO logic';
      ELSE INTERFACE.MOO 'Have to do MOO logic';
END

Now our sample code does not bounds to the actual business logic directly, instead it calls an abstracted logic through an interface (which we can override when needed), allowing the calling process to inject desired functionality.
Let’s do it…
Make a new object somewhere in another database:

CREATE PROCEDURE SOMEOTHERDB.MOCK_OBJECTS.BOO
@Message VARCHAR(50)
AS
BEGIN
   DECLARE @OverrideName VARCHAR(100)
     = 'SOMEOTHERDB.MOCK_OBJECTS.BOO'
   PRINT 'Very simple log message';
   INSERT INTO TEST_MESSAGES
      (OverrideName, [Message], EventDate)
      VALUES (@OverrideName,@TmpMessage, GETDATE());
END

Then (on your development server) change BOO synonym to point to the mocked object:

DROP SYNONYM INTERFACE.BOO;
CREATE SYNONYM INTERFACE.BOO
   FOR SOMEOTHERDB.MOCK_OBJECTS.BOO;

After this change, you can call business logic and verify that expected message got logged in to a test table.

--call logic
EXEC dbo.sp_BOO_manager;
--assert result
EXEC DBTD_ASSERT_COLUMN_HAVE_VALUE
   @v_TableName = 'TEST_MESSAGES',
   @v_ColumnName = 'OverrideName',
   @v_Value = 'SOMEOTHERDB.MOCK_OBJECTS.BOO',
   @v_ValueDataType = 'VARCHAR(100)',
   @v_UserMessage = 'BOO functionality should run';

There are a few ways to introduce dependency injection pattern in databases, we have chosen SYNONYMS, because of the following benefit:

  • Synonyms are very simple database objects;
  • Synonym does not change actual business logic they represents;
  • Synonym provides less invasive way to perform injection;
  • When creating synonym developer only need the name of the synonym (already known from consumer side) and the name of the object it represents (which is also known by developer from a provider side );
  • Locating synonyms and business logic in the different schemas allows synonym to have the same name as the object it abstracts, this preserve an existing naming convention used by development team;
  • Synonym can be created on the following objects:
    CLR Stored Procedures, Table-valued Functions, Scalar Functions, and Aggregate Functions, SQL Stored Procedure, Replication-filter-procedures, Extended Stored Procedures, SQL Scalar Functions, SQL Table-valued Functions, SQL Inline-table-valued Function, Views, User-defined Tables, includes local and global temporary tables

Well, here you go, now you have one more tool in your development toolbox. Use injection wisely, make sure you have basic infrastructure tests to go with it, and don’t forget about security.

Planning a Customer Product Roll-Out?

7 Jan

First, Walk in Their Shoes

(So you don’t have to swim in them)

Joseph Konrad
Senior Business Analyst

titanic

When the Titanic’s captain was asked how her maiden voyage would go, he reputedly said, “Swimmingly.” Her construction was considered the acme of civilian ship architecture, and her voyage in 1912 carried a Who’s Who of high-society Americans. (Many had been in Britain for a year experiencing the glamour of a coronation and also, more importantly, marrying impoverished nobility willing to confer a title in return for refilling the empty family coffers.) This new maritime product’s ‘rollout’ was to be every bit as confident as its construction, expressing an ideal captured in the Japanese word ‘shibumi’, a movement of effortless perfection. Everything would perform well by its nature.

Instead, in the epitome of disaster, we got a Leonardo DiCaprio film.

Step One: Boring, Methodical Preparation

Success may be so uneventful as to look easy, even banal, but failure can be spectacular. Given the choice, I’ll take banality.

In creating a product and introducing it to its customer, a desire for effortless perfection can’t hide that we live in the world of Murphy’s Law, and supreme confidence is no legal defense. Reality demands another approach at every step of an endeavor, Rigor. It may not result in even the appearance of ‘shibumi’, but it can prevent us from going to the bottom. This form of QA may be doggedly dull, but it’s also essential in the process of successfully creating and deploying a product to a customer.

This imperative was on display during the spring, summer and fall of 2012, when, at the request of one of our most important and growing relationships, Fortigent expanded its suite of reports to include more fixed income data, income projections for both bonds and equities, and the use of sector/industry data for equities.

When the new reports were being developed, numerous constituent parties had to be identified in their relationships and organized. Within Fortigent, this endeavor included our consultants interacting with the outside customer. It needed people with industry and business analysis knowledge (helping to design specifications) and developers (creating the new reports). Expansive integration testing meant growing the test team beyond its normal core of professionals to include consultants, Operations team members, developers and analysts. The final test was (and remains) the ultimate user acceptance test, by the customer.

Through it all, we worked to follow an orderly plan (tailored as events required) and to keep constituents positively engaged. We had to know who the customers were for each component. If the report developer needed improved report requirements, she was my customer and it was my job to provide them. To the suppliers of vital meta-data that feeds these report beasts, it’s we who are the customers. To the developer of the data integrity controls that helps Operations shoulder responsibility for the data garden, Operations was the customer.

Step Two: Boring, Methodical Expansion

Our initial thrust was outward, to one customer. Now these reports will be made available to another of our largest customers. It’s become a redeployment process.

On this second cruise we start by identifying the prize: As trouble-free a roll-out of this new product to a customer as possible. A satisfying experience includes keeping the customer engaged, educated, and happy with their initial and subsequent experiences of the new tools.

How do we ensure that last bit? By pretending to be the customer ourselves and living their experience through a dry-run, by taking a walk in their shoes. We identified 20 of the most sizable composite portfolios for which these reports will be run, created sample report packages for them, and started running them as of a recent date. We surveyed the quality of the meta-data associated with the securities. Were their ratings ready? Their effective durations? Was call schedule data up to date?

The first reports were surprisingly good, but had rough edges. Where we found gaps, we filled them, and simultaneously improved our internal QA tools to keep them filled. After upgrades, the second batch was better, and we started looking for the non-obvious at the granular level. Starting big, we worked our way down into the nitty-gritty of the customer experience. Anything we found benefited not only this second customer, but meant an incremental gain for the first as well. If you have to make mistakes, never make the same one twice. Make educational new ones.

Soon it will be time to bring the customer in more directly. We’ll prepare a real-world demonstration with real client data and engage in energetic interaction, fielding questions, capturing ideas, and creating expectations both realistic and positive. Finally, as confidence in data quality and its utilization rises, there will be more formal training and education: Here’s how this works. What do you think? What do you need? Can we make it better? The second customer, like the first, will be a partner, not a passenger.

In effect we want the go-live date for our customer to be ho-hum. In fact, if we have done our part rigorously, they will have been effectively ‘live’ without knowing it even before the roll-out date.

Step Three: Boring, Methodical Follow-through

Something unexpected will always happen. It probably won’t be a shipwreck, but internal redundancy expects – in fact, demands – every effort be made to find problems with data quality or utilization long before an external customer might. And if snags are found, it means having a trained, experienced team to work the problem, inform the customer, gather feedback, and energize the solution at every step of their experience.

These are living products and long-term commitments. They don’t get sold, then forgotten. Each successive deployment, each ‘sale’, becomes easier until the new feature is part of the essential fabric of our product offering.

Methodical analysis. Thorough planning. Rigorous multi-disciplinary testing. Flexible thinking and resource allocation. Customer engagement. Committed, energetic follow-up.

These are all essentials in the rigorous alchemy of customer satisfaction. It may not be effortless perfection – these things take work – but it can be a comfortable and profitable non-event, free of icebergs.


Joseph KonradJoseph Konrad started at Marine Midland Bank (extinct) as a trust assistant. From there
he worked in portfolio accounting, performance measurement, trading and operations
management before joining Fortigent as a
business analyst.

Product Ownership in an Agile Environment

4 Dec

The technology group at Fortigent is currently undergoing some reorganization around a new group of ‘product owners’ for the various software solutions and initiatives we are working on.

In software development, the product owner’s typical responsibilities revolve around setting overall direction and ensuring that all stakeholder’s needs are addressed.  This usually involves building up requirements and setting priorities between competing features and  internal design enhancements.

But product ownership in an agile environment can be different from what people are typically accustomed to. In this system, which emphasizes rapid response to change and communication, you need to set up an iterative development cycle, with robust communication between groups, and mostly get out of the way.

It’s your job, in many ways, to simply foster the right culture. In many respects, the product is collectively ‘owned’ rather than a traditional top-down dictatorial waterfall approach.

In that vein I will layout what, in my opinion, are three most important duties of the agile product owner:

Communication

Individuals and interactions over process… Customer collaboration over contract negotiation.  This is the agile manifesto.  It’s not good enough to say ‘we gave you what you asked for’ anymore. Agile development is iterative, with constant feedback loops.  You must know as early as you can when something isn’t going the right direction. You do this with demos. You do it with conversations… phone calls… meetings.

This communication is NOT uni-directional.  It’s not just stakeholders feeding their wants and needs into engineers.  Engineers feed wants, needs and ideas to stakeholders also.  The best software comes from constant communication between these groups. Get the groups together. Encourage them to talk.  Don’t say ‘I’ll talk to Dan and get back to you’.  Instead, say ‘why don’t you go ask Dan about that’ or ‘let’s call Dan over and talk about it’. The better the engineers understand the business, and the better the business understands the engineers, the better the product.

Prioritization

You’ve got to be able to respond to change in an agile environment and prioritization is a constant battle. You need to have a sense of what’s most important, knowing that priorities will change and change often. Don’t get too caught up in the details.  Pick something to work on next and move on.  No one will lose any sleep if you work on A and then B… when it should have been B first. Don’t over-plan… If you trying to plan out all the way to Z you are, quite literally, wasting everyone’s time.

Recognize when engineers are going down what we call a ‘rabbit hole’… taking on a purely technical change that has no business value that takes too much time. Sometimes you have to do those things and make those choices, but sometimes you’ve bitten off more than you could chew.  These are typically difficult decisions to make.  See if you can make these changes incrementally, spreading them over multiple iterations. Involve people in the decisions. Give the business a list and say ‘one feature needs to go’ and figure out what that is. Tell the engineers when something must be released at the end of the month and they need to cut some corners. Let them figure out what can be simplified.

Innovation

This is one thing the Agile Manifesto doesn’t really address when, to me, the whole business of software development is about ideas. Finding clever solutions to difficult problems. Coming up with a new way to do things that no one thought of before. It’s about creativity. Who wants to sell the same thing three other companies are selling? The name of the game is innovation.

As important as it is to listen to your stakeholders, however, they shouldn’t design your product. Sometimes the clients don’t know what they want until you show it to them. Bring your own ideas to the table, listen to your engineers and anyone else in the room. What more could the product be above and beyond what everyone has accepted as ‘the product’? Encourage this kind of thinking in the group. Don’t bring solutions to the group, bring problems. Get feedback from your end users and read it. If one thing doesn’t work, try something else.

Above all else, don’t stand still… you’ve got to move, man!  That’s what agile is all about!