CU Employee CULytics Founder

Data Warehouse Development Options

Data warehouse or data-lake or lake-house is a core component of your data analytics program. (In this article, I am using the term data warehouse in a generic sense) It pulls together data from internal applications/data-sources such as core banking, lending apps, origination, collections, marketing, finance, etc. as well as from second and third party apps into a single repository for critical analytics support decision-making.

 Here is a typical high level architecture of a data warehouse.

10829850291?profile=RESIZE_710x

                                                                                                   Source - https://www.ibm.com/cloud/learn/data-warehouse

 

A typical data warehouse requires a database, a data migration or ETL or ELT tool, and a business intelligence tool to build reports and dashboards. When looking to get a data warehouse, one big decision needs to be made early on, which is whether you want to create a data warehouse or buy one.

Best-of-breed Solution

When you want to build your data warehouse, you get best-of-breed technologies and architect, design, and develop independently. You have total flexibility regarding what data you want to bring to the data warehouse, how you organize it, what reports and dashboards you want to build, or what AI/ML models your business wants. Of course, to do all this, you need a talented team. e.g. of best of breed technologies include Snowflake, Tableau, PowerBI, Talend, Qlik, etc.

Full Service Solution

These days, some vendors have credit union industry-specific data warehouses. Let's call them full-service data warehouse solutions to differentiate them from the best-of-breed data warehouse. These solutions include a preselected database, a data migration tool, and a business intelligence tool. To add value to their solution, they have built data ingestion from commonly seen applications such as core banking, loan origination systems, and digital banking. Etc. They continue adding more data sources to their solutions, from where they can seamlessly ingest data. The solutions also include industry-specific schema and pre-built data migration and transformation pipelines. Some solutions have added a bunch of data quality checks and industry-specific business rules to these data pipelines to add value to their solution further. These solutions include dozens of out-of-box reports, dashboards, and AI/ML-based models. e.g. of full-service solutions include Aunalytics, Tibco/ADMS, Arkatecture, Trellance/M360, VeriCU, etc. to name a few.

Comparison 

Both these approaches are very distinct and have their pros and cons.

The best-of-breed solution offers you maximum flexibility but requires significant effort up front in procuring the right set of technologies, installing them, and then administrating and operating them. All along, you need to have a reasonably good understanding of the best practices, define and document the design and standards, set up processes, etc. Significant advance thinking and work goes behind setting up a data management solution using best-of-breed technologies. When organizations select best-of-breed technologies and start the development with little or no design and standards for their architecture, design, development, operations, etc. then may see some early results but are very likely to have challenges in scaling the program from a long-term perspective. Any flawed design decision made in the process is likely to have significant long-term impact on the overall program.

In contrast, full-service solutions give your data analytics program a jumpstart with a preselected database, data migration, and business intelligence. All these technologies are integrated and work well together. However, these solutions do come at a cost. Also, you are now dependent on your solution provider roadmap for solution capabilities that may be critical to you. e.g., The ability to ingest data from an application the solution does not ingest from today or a transformation, business rules, or reports/dashboards that may be critical for your business.

When organizations select any solution without well-defined requirements then they are likely to miss critical success factors out as part of the evaluation process. This may have a substantial impact on the value that the program is expected to deliver as there are exorbitant switching costs.

These two approaches can be seen as two ends of a spectrum, with quite some options for you in the middle. You can decide to go either way or take a hybrid approach. Some vendors also allow you to take a full-service approach to jumpstart your data analytics program and then let you build from there. In that situation, you are essentially committing to their preselected data warehouse technologies, data warehouse schema, development standards for data pipelines, and business intelligence.

The decision to come up with the right approach to build your data warehouse is very personal and strategic. It is very much tied to your credit union's vision, strategy, scale, and data analytics talent, among other things. One thing for sure is that whatever decision you make has some long-term implications for the actual cost, opportunity cost, data analytics team, and overall success of the data analytics program at your credit union. Taking an incorrect decision may lead to a higher overall cost of the solution, longer timelines to get value from the investment, unsatisfied stakeholders, and missed expectations. Therefore, this decision should be taken with appropriate caution.

E-mail me when people leave their comments –

You need to be a member of CULytics Community to add comments!

Join CULytics Community

 

advantedge
altair
ibi
arka
trellance
coopfs
dfa
wherescape
alkami
prismacampaigns
marquis
aiq
totex
cnet
datava
aun
cinch
know

Related Post

 

Ad Unit Settings






Ad Url Settings

 

api-lead-approach
the-amazon-lending-experience
executing-advanced-analytics-do-s-and-don-t
lending-transformation-old-vs-new
data-journey-building-strong-analytical-practices
4-step-iterative-process-building-a-relevant-analytics-practice
significant-measures-towards-new-normal
building-a-strong-analytics-practice-recipe-for-success
data-warehouse-evaluation-and-implementation
explainable-ai-trust-and-transparency
forecasting
top-50-members-using-transactional-website-jun-2020
top-50-cus-with-highest-and-lowest-efficiency-june-2020
importance-of-financial-risk-management
secret-sauce-for-long-term-sustainable-business-intelligence-succ
top-pfm-technologies
secret-sauce-for-long-term-sustainable-business-intelligence-succ
top-pfm-technologies
data-warehouse-and-bi-technologies-opportunities-challenges
top-chatbot-technologies
keys-to-building-an-effective-branch-or-atm-network
top-50-credit-unions-with-highest-and-lowest-accounts-per-member
lowest-and-highest-net-income-per-branch
marketing-holy-grail
top-50-most-and-least-delinquent-credit-unions
modern-marketing-technologies
incremental-low-cost-data-driven-wins
power-of-storytelling
the-cost-of-not-investing-in-data-governance
questions-you-should-ask-before-investing-in-data-warehouse
learnings-from-new-data-based-on-auto-loan-pricing
5-questions-you-need-to-ask-before-investing-in-data-governance
digital-marketing-maturity-models-for-credit-unions
marketing-expense-per-member
top-2-reasons-that-are-holding-credit-unions-back-when-they-are-i
using-data-analytics-to-manage-lending-complexity-while-driving-h
5-reasons-your-credit-union-should-invest-in-data-and-digital-now
top-50-most-and-least-efficient-credit-unions
retail-financial-services-outlook-during-covid-19
use-of-operational-analytics-to-mitigate-the-impact-of-covid-19
top-50-credit-unions-based-on-asset-size
cu-peer-comparison-dashboard
cu-peer-benchmark
all-about-machine-learning-engineering
top-web-design-trends
most-important-social-media-marketing-trends
state-of-digital-marketing-maturing-in-credit-unions
top-kpis-for-email-marketing
data-cloud-and-the-digital-transformation-imperative
digital-trinity-and-you
phases-of-financial-industry
analytics-roundtable-workshop
invitation-to-join-digital-transformation-hub
analytics-in-the-credit-union-business
value-of-member-centricity-and-analytics-in-the-growth-of-cus
all-about-membership-analytics
top-fraud-management-technologies
getting-started-with-your-data-analytics-journey
explore-vizualization-for-credit-unions
investment-in-website-personalization-technologies
data-analytics-supporting-cu-s-first-member-philosophy
loyalty-rewards-and-retention-technologies
member-experience-analytics
channel-analytics-and-its-importance
project-portfolio-management-technologies
investment-in-self-service-data-preparation-technologies
self-service-data-preparation-technologies
new-frontier-in-customer-experience-management
role-of-marketing-analytics-in-credit-unions
important-aspects-of-consumer-lending-analytics
kpis-on-website-analytics
journey-towards-bank-less-banking
investment-in-crm-technologies
top-omni-channel-vendors
conversational-banking-solutions
/top-kpis-for-chief-information-officer
mistakes-to-avoid-when-implementing-a-omnichannel-member
top-things-to-consider-when-building-dashboards
making-digital-marketing-more-agile-through-tag-managers
cecl-solution-providers
mistakes-to-avoid-while-implementing-marketing-automation
p2p-payment-integrated-solutions
kpis-for-social-media-tracking
kpis-for-human-resources-management
investment-in-fintechs-should-or-should-not
top-kpis-for-online-banking
investment-in-marketing-automation-technologies
investment-in-e-signature-technologies-should-or-should-not
tips-and-tricks-to-a-successful-bi-program
kpis-for-credit-card-business
kpis-for-digital-marketing
kpis-for-consumer-lending
hot-topics-for-credit-union-data-leaders
kpis-for-debt-collections
kpis-for-finance
website-personalization-tools
data-integration-technologies
robotic-process-automation-tools
why-data-analytics-initiatives-fail
electronic-signature-softwares
data-governance-tools-for-credit-unions
digital-and-mobile-banking-technologies
report-inconsistencies-are-frustrating
is-your-culture-ready-for-data-analytics
three-big-data-myths
turning-transaction-data-into-a-goldmine-a-becu-case-study
call-for-presentation-for-2019-credit-union-analytics-summit-is-n
top-10-keys-to-successful-data-analytics-practice
credit-union-chooses-accountscore-for-open-banking-transaction-da
how-much-do-you-spend-to-serve-a-customer
marketing-automation-technologies-for-credit-union
alexa-ask-first-abilene-fcu-for-my-balance
dataweb-content-management-technologies-for-credit-unions
efficiency-ratio
web-analytics-technologies
data-warehousing-software-for-banks
customer-experience-software
the-best-kept-secret-for-credit-union-data-analytics
mark-sievewright-on-technology-trends
naveen-jain-on-credit-union-analytics-summit-2018
why-analytics-doesn-t-make-a-difference-by-gary-angel
cuas2018-harnessing-the-right-data
build-a-financial-phone-assistant-for-your-credit-union-in-3-step
2018-culytics-analytics-challenge-winner
update-from-naveen
error-resolution
benefits-of-conversational-apps
who-are-your-most-valuable-members-part-1
how-alexa-can-help-your-credit-union
top-10-kpis-for-measuring-retail-channel-performance
how-much-is-too-much-personalization
top-10-kpis-for-measuring-contact-center-efficiency
pressure-on-margins-for-auto-loans-indirect-auto-loans-declining
best-business-intelligence-technologies-for-credit-unions
establishing-a-thriving-data-analytics-practice-is-a-journey
educational-presentations-from-the-2017-axfi-conference
modelling-alternatives-for-cecl-a-deep-future-analytics-study
data-analytics-use-cases-for-credit-unions-infographic
data-analytics-opportunities-in-credit-union-business
loan-application-analytics-with-cufx
machine-learning-delivers-great-consumer-experiences
deep-insights-of-credit-union-members-data-with-machine-learning
web-analytics-reporting-tips-for-credit-unions
big-data-strategy-roadmap-our-data-journey
webinar-framework-for-member-focused-decision-making
too-many-regulations-hurt-credit-union-members
digital-marketing-automation-solutions
online-banking-boom
transformation-transactions-to-relationships
top-dispute-management-technologies
2020-retail-trends
future-of-artificial-intelligence
2020-culytics-summit-attendee-dashboard
repositioning-the-role-of-marketing
marketing-automation-a-step-towards-marketing-transformation
strategic-agility
using-data-to-navigate-through-the-new-normal
digital-transformation-bcu
highest-and-lowest-new-loan-balances-per-branch-as-of-jun-2020
-new-members-ratio-as-of-june-2020
cus-with-highest-and-lowest-loan-grants-per-member-june-2020
self-service-data-preparation-technologies
highest-and-lowest-marketing-expense-per-member-june-2020
the-amazon-lending-experience
api-lead-approach
4-step-iterative-process-building-a-relevant-analytics-practice
data-journey-building-strong-analytical-practices
post-election-the-cu-outlook
most-and-least-delinquent-credit-unions-sept-2020
leveraging-ach-data-to-produce-real-outcomes
member-engagement-scores-benefits
member-engagement-key-to-serve-the-best
story-of-james-an-intelligence-transformation
executive-kpis-the-pulse-of-the-organization
untangling-member-journey
onboarding-strategy-to-deliver-success
the-importance-of-digital-technologies
top-interactive-financial-calculators
using-artificial-intelligence-to-improve-your-productivity
organizational-transformation-to-drive-growth
multi-year-journey-through-data-transformation
top-50-cus-with-the-highest-and-lowest-member-per-branch
digital-transformation-lessons-through-the-eyes-of-a-ceo
organizational-readiness-for-digital-transformation
ruthless-prioritization-to-do-more-to-learn-more-and-to-earn-more
performance-measures-for-digital-services
analytical-maturity-journey-towards-growth
less-is-more-the-necessity-of-focus-for-strategic-success
solving-the-crm-mrm-puzzle
insights-driven-messaging-member-and-product-onboarding
performance-measures-for-marketing
data-insights-that-drive-member-product-innovation
solving-the-crm-mrm-puzzle
the-agility-flywheel-a-strategy-that-never-goes-out-of-the-way
artificial-intelligence-as-a-playing-field-for-credit-unions
performance-measures-for-call-centers
top-automl-technologies
performance-measures-for-lending
building-business-case-for-data-analytics
driving-innovation-and-change
data-analyze-decide-and-create
digital-readiness-important-steps-to-achieve
digital-readiness-important-steps-to-achieve
enabling-credit-unions-with-ai
culytics-virtual-summit-2022-a-resounding-success
culytics-virtual-summit-2022-day-1
digital-banking-roundtable
digital-marketing-roundtable
transformative-lessons-from-a-chief-digital-officer
data-analytics-roundtable-mar-11
rewind-2022-culytics-day-key-highlights
data-analytics-team-roles
data-warehouse-development
data-analytics-team-size
is-your-data-analytics-program-not-delivering-results
active-deposit-management-for-profitable-growth
data-modeling
maximize-your-success-with-2023-CULytics-summit
biggest-opportunities-for-credit-unions
should-ceos-attend-the-culytics-summit
the-cost-of-a-wrong-decision
biggest-roadblocks-in-becoming-data-driven
a-journey-for-all-organizational-maturity-levels
maximize-your-data-analytics-checkup
navigating-the-data-analytics-landscape
improving-data-literacy
why-credit-union-leaders-should-invest-in-their-teams
why-credit-unions-should-not-invest-in-building-predictive-models
why-should-measure-the-success-of-data-analytics-program
cost-of-choosing-the-wrong-data-analytics-technology-stack
why-data-analytics-strategy-focus-on-supply-and-demand-side
kpis-to-measure-the-success-of-data-analytics-program
data-analytics-for-credit-union-branch-heads
data-organizing-principles
top-data-warehouse-storage-technologies
discover-the-hidden-truth-behind-watermelon-kpis
unveiling-the-hidden-dangers-of-cobra-effect-on-kpis
are-you-accurately-interpreting-your-kpi
unmasking-biases-a-guide-to-data-analysis-and-kpi-definition
uncover-the-power-of-proxy-kpis
unraveling-the-hidden-impact-of-sampling-bias-in-credit-unions
bi-department-structure
hidden-impact-of-confirmation-bias-in-credit-unions
getting-executive-attention-for-your-data-analytics-program
uncovering-biases-in-data-preprocessing
navigating-missing-data-in-credit-unions
navigating-sampling-bias-in-cu
unleash-the-power-of-real-time-data-use-cases
how-confirmation-bias-impacts-cus
breaking-down-selection-bias-in-credit-unions
unmasking-reporting-bias
elevate-your-cu-with-data-analytics-expertise
understanding-and-tackling-volunteer-bias-in-credit-unions
time-period-bias-in-credit-union
overcoming-biases-in-credit-unions
embracing-the-future-fast-future-fundamentals-program-equips-cred
unlock-growth-and-efficiency-credit-unions-guide-to-generative-ai
how-better-data-and-behavioral-biometrics-can-help-credit-unions-
harnessing-the-power-of-data-in-credit-unions
leveraging-third-party-data-a-strategic-guide-for-credit-unions
unlocking-member-insights-how-cus-can-leverage-third-party-data
enhancing-customer-experience-through-third-party-data
third-party-data-integration-techniques-and-technologies
the-future-of-lending-third-party-data-role-in-credit-decisioning
how-third-party-information-shapes-cu-strategies
using-data-to-improve-access-to-credit-for-low-income-members
designing-financial-products-for-low-income-members-using-data
measuring-and-enhancing-the-impact-of-support-programs
data-governance-why-selling-internally-is-important
selling-data-governance-in-your-credit-union
building-a-business-case-and-engaging-stakeholders
creating-a-data-governance-roadmap-and-executing-it
measuring-and-demonstrating-the-impact-of-data-governance
sustaining-momentum-keeping-data-governance-a-priority
overcoming-challenges-in-transaction-data-analysis-credit-unions
empowering-members-through-transaction-data
how-credit-unions-leverage-transaction-data-best-practices
unlocking-financial-independence-the-power-of-transaction-data
the-power-of-transaction-data-enrichment
avoid-financial-reputation-and-member-trust-issues
introduction-to-model-risk-management
week-1-mrm-a-practitioner-s-approach
week-2-guide-to-identifying-and-maintaining-models
survey-insights-navigating-mrm-in-credit-unions
week-3-application-of-mrm-insights-to-sound-model-development-eff
unlocking-the-secrets-to-attracting-gen-y-and-z
creating-a-seamless-member-experience-for-gen-y-and-gen-z
data-analytics-maturity-assessment-report
marketing-to-gen-y-and-z-strategies-that-work-for-credit-unions
the-imperative-of-engaging-millennials-and-gen-z
cu-build-lasting-relationships-with-gen-z-financial-literacy
how-social-responsibility-drives-gen-z-membership
loyalty-programs-that-work-keeping-gen-y-and-z-members-engaged
insights-on-engaging-millennials-and-gen-z-at-credit-union
ai-driven-member-experience
streamlining-operations-with-ai
innovation-and-member-inclusion-in-ai-credit-risk-models
ai-risk-management-enhancing-fraud-detection-and-cybersecurity
how-ai-is-transforming-data-analytics-for-credit-union
overcoming-ai-adoption-challenges-in-credit-unions
the-state-of-ai-in-credit-unions-survey-insights
creating-a-culture-of-innovation
building-the-foundation
closing-the-talent-gap
ensuring-data-readiness
navigating-the-roadblocks-ai-and-data-analytics-in-credit-unions