{"id":736,"date":"2016-08-17T15:52:17","date_gmt":"2016-08-17T15:52:17","guid":{"rendered":"http:\/\/zappysys.com\/blog\/?p=736"},"modified":"2024-07-10T21:06:01","modified_gmt":"2024-07-10T21:06:01","slug":"aggregate-mongodb-data-using-group-by-query-in-ssis","status":"publish","type":"post","link":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/","title":{"rendered":"How to write MongoDB Aggregation Queries in SSIS (Group By)"},"content":{"rendered":"<h2>Introduction<\/h2>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2115 alignleft\" src=\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2017\/08\/mongodb-logo.png\" alt=\"\" width=\"88\" height=\"88\" \/>In this post you will learn how to write flexible <strong>MongoDB Aggregation Queries in SSIS<\/strong> (i.e. Group By Query) using <a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/ssis-mongodb-source\/\" target=\"_blank\" rel=\"noopener\">SSIS MongoDB Source Component<\/a>. MongoDB provides powerful Aggregation Pipeline Engine which is conceptually same as writing Group By queries in traditional SQL world but its not exactly same.\u00a0<a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/ssis-mongodb-source\/\" target=\"_blank\" rel=\"noopener\">SSIS MongoDB Source<\/a> returns nested MongoDB JSON documents into flat looking table with rows and column (denormalized data).<\/p>\n<p>If you want to learn <a href=\"https:\/\/zappysys.com\/blog\/ssis-loading-data-into-mongodb-upsert-update-delete-insert\/\" target=\"_blank\" rel=\"noopener\">how to write data to MongoDB (Insert, Update, Delete, Upsert) then click here<\/a>. If you want learn <a href=\"https:\/\/zappysys.com\/blog\/read-mongodb-data-from-array-extract-nested-sub-documents\/\" target=\"_blank\" rel=\"noopener\">how to read data from MongoDB then click here<\/a>.<\/p>\n<p>So let&#8217;s explore concepts of\u00a0<a href=\"https:\/\/docs.mongodb.com\/manual\/core\/aggregation-pipeline\/\" target=\"_blank\" rel=\"noopener\">MongoDB Aggregation Pipeline<\/a> and see some MongoDB query examples for SSIS use case.<\/p>\n<div class=\"content_block\" id=\"custom_post_widget-2523\"><h2><span id=\"Prerequisites\">Prerequisites<\/span><\/h2>\r\nBefore we perform the steps listed in this article, you will need to make sure the following prerequisites are met:\r\n<ol style=\"margin-left: 1.5em;\">\r\n \t<li><abbr title=\"SQL Server Integration Services\">SSIS<\/abbr> designer installed.\u00a0Sometimes it is referred to as <abbr title=\"Business Intelligence Development Studio\">BIDS<\/abbr> or <abbr title=\"SQL Server Data Tools\">SSDT<\/abbr> (<a href=\"https:\/\/docs.microsoft.com\/en-us\/sql\/ssdt\/download-sql-server-data-tools-ssdt\" target=\"_blank\" rel=\"noopener\">download it from the Microsoft site<\/a>).<\/li>\r\n \t<li>Basic knowledge of SSIS package\u00a0development using\u00a0<em>Microsoft SQL Server Integration Services<\/em>.<\/li>\r\n \t<li>Make sure\u00a0<span style=\"text-decoration: underline;\"><a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/\" target=\"_blank\" rel=\"noopener\">ZappySys SSIS PowerPack<\/a><\/span>\u00a0is installed (<a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/download\/\" target=\"_blank\" rel=\"noopener\">download it<\/a>, if you haven't already).<\/li>\r\n \t<li>(<em>Optional step<\/em>)<em>.<\/em>\u00a0<a href=\"https:\/\/zappysys.zendesk.com\/hc\/en-us\/articles\/360035974593\" target=\"_blank\" rel=\"noopener\">Read this article<\/a>, if you are planning to deploy packages to a server and schedule their execution later.<\/li>\r\n<\/ol><\/div>\n<h2>SSIS MongoDB Source Introduction<\/h2>\n<p><a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/ssis-mongodb-source\/\" target=\"_blank\" rel=\"noopener\">SSIS MongoDB Source<\/a>\u00a0supports three modes to read data from MongoDB collection. Each mode also supports <a href=\"https:\/\/zappysys.com\/blog\/read-mongodb-data-from-array-extract-nested-sub-documents\/\" target=\"_blank\" rel=\"noopener\">JSONPath Expression to extract nested array<\/a>.<\/p>\n<ol>\n<li>Table Mode<\/li>\n<li>SQL Query Mode<\/li>\n<li>JSON Query Mode (Native)<\/li>\n<\/ol>\n<p>To read more about supported SQL Syntax read <a href=\"https:\/\/zappysys.com\/onlinehelp\/ssis-powerpack\/scr\/mongodb-source.htm\" target=\"_blank\" rel=\"noopener\">help file here<\/a>.<\/p>\n<h2>Using SSIS MongoDB Source to Query Data (Aggregation)<\/h2>\n<p>Now lets look at how to use <a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/ssis-mongodb-source\/\" target=\"_blank\" rel=\"noopener\">SSIS MongoDB Source<\/a>\u00a0to query MongoDB data using Aggregation Engine.<\/p>\n<h3>MongoDB Aggregation Pipeline Syntax in SSIS<\/h3>\n<p><a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/ssis-mongodb-source\/\" target=\"_blank\" rel=\"noopener\">SSIS MongoDB Source<\/a>\u00a0support all pipeline commands to write aggregation queries but its slighly different syntax than shell command so your SHell Query won&#8217;t work directly but you can copy some part of your Shell query. Check below Syntax.<\/p>\n<p><strong>MongoDB Native Query Syntax (Shell Command) for Aggregation:<\/strong><\/p>\n<pre class=\"lang:default decode:true \">db.YourCollectionName.aggregate(\r\n[\r\n[ &lt;pipeline-stage1&gt;]\r\n[,&lt;pipeline-stage2&gt;]\r\n...\r\n...\r\n[,&lt;pipeline-stageN&gt;]\r\n]\r\n)<\/pre>\n<p><strong>SSIS MongoDB Source Query Syntax for Aggregation:<\/strong><code><br \/>\n<\/code><\/p>\n<pre class=\"lang:default decode:true \">{Table:YourTableName ,Method:aggregate [,MaxRows: N] [,AllowDiskUse:true] [,AllowCursor:true] [,Timeout:#####] }\r\n{\r\npipeline: [\r\n[ &lt;pipeline-stage1&gt;]\r\n[,&lt;pipeline-stage2&gt;]\r\n...\r\n...\r\n[,&lt;pipeline-stageN&gt;]\r\n]\r\n}<\/pre>\n<p>&nbsp;<\/p>\n<h2>Example Aggregation Queries (SQL to SSIS Mapping)<\/h2>\n<p>Assume that we want to write following SQL Query using <a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/ssis-mongodb-source\/\" target=\"_blank\" rel=\"noopener\">SSIS MongoDB Source<\/a>\u00a0or using MongoDB Shell<\/p>\n<pre class=\"codeblock\">SELECT State as StateAbbr,SUM(population) as TotalPopulation \r\nFROM ZipCodes \r\nWHERE status='Active' \r\nHAVING SUM(population) &gt; 25\r\nOrder By TotalPopulation DESC<\/pre>\n<p>Above SQL Query can be translated to below JSON Query which can be used in SSIS MongoDB Source. As we noted before Below syntax is not same as Shell command so it will only run in SSIS MongoDB Source.<\/p>\n<pre class=\"codeblock\">{Table: ZipCodes,Method: aggregate}\r\n{\r\n  pipeline: \r\n  [\r\n    {$match : { status: \"A\" } } ,\r\n    {$group : {_id :  \"$state\", TotalPopulation: { $sum: \"$pop\" }}},\r\n    {$match: { TotalPopulation: { $gt: 25 } } },\r\n    {$project : { StateAbbr: \"$_id\" , _id: 0, TotalPopulation:1 } },\r\n    {$sort : {TotalPopulation : -1}}\r\n  ]\r\n}<\/pre>\n<p>So as you saw in above query various clause of SELECT queries are defined as pipeline stages. You can add any possible stages in different order. Based on their order it will be applied so order of stage is very important.\u00a0 For more information about various <a href=\"https:\/\/docs.mongodb.com\/manual\/reference\/operator\/aggregation-pipeline\/\" target=\"_blank\" rel=\"noopener\">pipeline stages click here<\/a>. Below is high level translation of SQL Query to MongoDB Aggregation Query.<\/p>\n<div style=\"width: 616px\" class=\"wp-caption alignnone\"><a class=\"figureimage\" href=\"https:\/\/zappysys.com\/onlinehelp\/ssis-powerpack\/scr\/images\/mongodb-source\/ssis-mongodb-group-by-query-aggregation-framework.png\" target=\"_blank\" rel=\"noopener\"><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/zappysys.com\/onlinehelp\/ssis-powerpack\/scr\/images\/mongodb-source\/ssis-mongodb-group-by-query-aggregation-framework.png\" alt=\"SSIS MongoDB Source - Example of MongoDB Group By Query (Aggregation Framework - Pipeline)\" width=\"606\" height=\"555\" \/><\/a><p class=\"wp-caption-text\">SSIS MongoDB Source &#8211; Example of MongoDB Group By Query (Aggregation Framework &#8211; Pipeline)<\/p><\/div>\n<p><strong>SQL to MongoDB Aggregation Mapping<\/strong><\/p>\n<div class=\"su-table su-table-alternate\">\n<table>\n<thead>\n<tr>\n<th>SQL Clause<\/th>\n<th>Mongo Pipeline Stage<\/th>\n<\/tr>\n<\/thead>\n<tbody>\n<tr>\n<td>SELECT<\/td>\n<td><a href=\"https:\/\/docs.mongodb.com\/manual\/reference\/operator\/aggregation\/project\/\" target=\"_blank\" rel=\"noopener\">$project<\/a><\/td>\n<\/tr>\n<tr>\n<td>WHERE<\/td>\n<td><a href=\"https:\/\/docs.mongodb.com\/manual\/reference\/operator\/aggregation\/match\/\" target=\"_blank\" rel=\"noopener\">$match<\/a><\/td>\n<\/tr>\n<tr>\n<td>GROUP BY<\/td>\n<td><a href=\"https:\/\/docs.mongodb.com\/manual\/reference\/operator\/aggregation\/group\/\" target=\"_blank\" rel=\"noopener\">$group<\/a><\/td>\n<\/tr>\n<tr>\n<td>HAVING<\/td>\n<td><a href=\"https:\/\/docs.mongodb.com\/manual\/reference\/operator\/aggregation\/match\/\" target=\"_blank\" rel=\"noopener\">$match<\/a><\/td>\n<\/tr>\n<tr>\n<td>ORDER BY<\/td>\n<td><a href=\"https:\/\/docs.mongodb.com\/manual\/reference\/operator\/aggregation\/sort\/\" target=\"_blank\" rel=\"noopener\">$sort<\/a><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n<p>Now let&#8217;s look at few examples how to translate traditional SQL\u00a0 Queries to JSON Query for SSIS MongoDB Source. We will use sample data we inserted in previous section.<\/p>\n<h3>Creating Sample Data for MongoDB Query Examples<\/h3>\n<p>For this exercise to write sample queries using below data. So let&#8217;s first create a collection called Order and insert following records using your favourite MongoDB client tool.<\/p>\n<pre class=\"lang:js decode:true\">db.Orders.insert(\r\n  {\r\n    \"OrderId\" : 1000,\r\n    \"CustomerId\" : \"CUS001\",\r\n    \"OrderTotal\" : 2000.50,\r\n    \"Quantity\" : 20,\r\n    \"OrderDate\" : ISODate(\"2010-01-01\"),\r\n    \"Country\" : \"USA\",\r\n    \"Items\" : [ {ItemId:10, UnitPrice: 100.00,Quantity: 10}, {ItemId:11, UnitPrice: 200.10,Quantity: 5}],\r\n    \"PaymentInfo\" : { \r\n      \"Method\" : \"Card\",\r\n      \"Currency\" : \"USD\" \r\n    }\r\n  }\r\n)\r\n\r\ndb.Orders.insert(\r\n  {\r\n    \"OrderId\" : 1001,\r\n    \"CustomerId\" : \"CUS001\",\r\n    \"OrderTotal\" : 1000.00,\r\n    \"Quantity\" : 10,\r\n    \"OrderDate\" : ISODate(\"2011-01-02\"),\r\n    \"Country\" : \"USA\",\r\n    \"Items\" : [ {ItemId:10, UnitPrice: 100.00,Quantity: 10}],\r\n    \"PaymentInfo\" : { \r\n      \"Method\" : \"Card\",\r\n      \"Currency\" : \"USD\" \r\n    }\r\n  }\r\n)\r\n\r\ndb.Orders.insert(\r\n  {\r\n    \"OrderId\" : 1002,\r\n    \"CustomerId\" : \"CUS002\",\r\n    \"OrderTotal\" : 1000.00,\r\n    \"Quantity\" : 5,\r\n    \"OrderDate\" : ISODate(\"2012-01-01\"),\r\n    \"Country\" : \"USA\",\r\n    \"Items\" : [ {ItemId:13, UnitPrice: 200.00,Quantity: 5}],\r\n    \"PaymentInfo\" : { \r\n      \"Method\" : \"Bank\",\r\n      \"Currency\" : \"USD\" \r\n    }\r\n  }\r\n)\r\n\r\ndb.Orders.insert(\r\n  {\r\n    \"OrderId\" : 1003,\r\n    \"CustomerId\" : \"CUS003\",\r\n    \"OrderTotal\" : 59000.00,\r\n    \"Quantity\" : 10,\r\n    \"OrderDate\" : ISODate(\"2005-01-02\"),\r\n    \"Country\" : \"India\",\r\n    \"Items\" : [ {ItemId:14, UnitPrice: 5900.00,Quantity: 10}],\r\n    \"PaymentInfo\" : { \r\n      \"Method\" : \"Check\",\r\n      \"Currency\" : \"INR\" \r\n    }\r\n  }\r\n)<\/pre>\n<p>Here is how it would look like (We used <a href=\"https:\/\/robomongo.org\/download\" target=\"_blank\" rel=\"noopener\"><strong>RoboMongo<\/strong><\/a> i.e. <strong>Robo 3T<\/strong>) but you can use mongo.exe command line to issue above commands.<\/p>\n<div id=\"attachment_2661\" style=\"width: 1110px\" class=\"wp-caption alignnone\"><a href=\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2016\/08\/ssis-mongodb-aggregation-queries-group-by-sql-pipeline-stages.png\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2661\" class=\"size-full wp-image-2661\" src=\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2016\/08\/ssis-mongodb-aggregation-queries-group-by-sql-pipeline-stages.png\" alt=\"Inserting Example data for MongoDB Aggregation Queries (RoboMongo \/ Robo 3T)\" width=\"1100\" height=\"677\" srcset=\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2016\/08\/ssis-mongodb-aggregation-queries-group-by-sql-pipeline-stages.png 1100w, https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2016\/08\/ssis-mongodb-aggregation-queries-group-by-sql-pipeline-stages-300x185.png 300w, https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2016\/08\/ssis-mongodb-aggregation-queries-group-by-sql-pipeline-stages-768x473.png 768w, https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2016\/08\/ssis-mongodb-aggregation-queries-group-by-sql-pipeline-stages-1024x630.png 1024w\" sizes=\"(max-width: 1100px) 100vw, 1100px\" \/><\/a><p id=\"caption-attachment-2661\" class=\"wp-caption-text\">Inserting Example data for MongoDB Aggregation Queries (RoboMongo \/ Robo 3T)<\/p><\/div>\n<h3>Example 1 &#8211;\u00a0MongoDB Aggregation Query with COUNT(*) &#8211; $group, $project<\/h3>\n<p>Here is how to write COUNT(*) query in MongoDB. Below query will find order count by country. Each query must output _id field and you can refer Document column using $ prefix (i.e. &#8220;<strong>$Country<\/strong>&#8221; or for nested object use dot separator &#8220;<strong>$PaymentInfo.Method<\/strong>&#8221; ) . To output _id column with different name we have used\u00a0Country: &#8220;$_id&#8221;. Because we including copy of _id we can remove _id column by doing\u00a0 <strong>_id: 0.<\/strong>\u00a0 To include column in output use <strong>YourColumanName: 1<\/strong>\u00a0in $project stage.<\/p>\n<p><strong>SQL Query:<\/strong><\/p>\n<pre class=\"lang:tsql decode:true\">SELECT Country,Count(*) as OrderCount \r\nFROM Orders\r\nGROUP BY Country<\/pre>\n<p><strong>Mongo Shell Query:<\/strong><\/p>\n<pre class=\"lang:default highlight:0 decode:true\">db.Orders.aggregate(\r\n [\r\n   {$group : {_id : \"$Country\",OrderCount: { $sum: 1 }}} \r\n  ,{$project : {Country: \"$_id\", _id: 0, OrderCount: 1 }}\r\n ]\r\n)<\/pre>\n<p><strong>SSIS MongoDB Source Query:<\/strong><\/p>\n<pre class=\"lang:default highlight:0 decode:true\">{Table: Orders,Method: aggregate} \r\n{ pipeline: \r\n   [\r\n    {$group : {_id : \"$Country\",OrderCount: { $sum: 1 }}} \r\n   ,{$project : {Country: \"$_id\", _id: 0, OrderCount: 1 }}\r\n  ]\r\n}<\/pre>\n<h3>Example 2 &#8211;\u00a0MongoDB Aggregation Query with WHERE and ORDER BY- $match, $group, $project, $sort<\/h3>\n<p>Here is how to write COUNT(*) query in MongoDB. Below query will find order count by country.<\/p>\n<p><strong>SQL Query:<\/strong><\/p>\n<pre class=\"lang:tsql decode:true\">SELECT Country,Count(*) as OrderCount \r\nWHERE Quantity &gt; 1\r\nFROM Orders\r\nGROUP BY Country\r\nORDER BY Country<\/pre>\n<p><strong>Mongo Shell Query:<\/strong><\/p>\n<pre class=\"lang:default highlight:0 decode:true\">db.Orders.aggregate(\r\n [\r\n   {$match : {Quantity : { $gt: 1 }}} \r\n  ,{$group : {_id : \"$Country\",OrderCount: { $sum: 1 }}} \r\n  ,{$project : {Country: \"$_id\", _id: 0, OrderCount: 1 }}\r\n  ,{$sort : {Country: 1}}\r\n ]\r\n)<\/pre>\n<p><strong>SSIS MongoDB Source Query:<\/strong><\/p>\n<pre class=\"lang:default highlight:0 decode:true\">{Table: Orders,Method: aggregate} \r\n{ pipeline: \r\n   [\r\n     {$match : {Quantity : { $gt: 1 }}} \r\n    ,{$group : {_id : \"$Country\",OrderCount: { $sum: 1 }}} \r\n    ,{$project : {Country: \"$_id\", _id: 0, OrderCount: 1 }} ,{$sort : {Country: 1}}  \r\n   ]\r\n}<\/pre>\n<h3>Example 3 &#8211;\u00a0MongoDB Aggregation Query with WHERE and HAVING Clause &#8211; $match, $group<\/h3>\n<p>Here is an example of WHERE and HAVING clause. They both use $match stage but their orders are different. WHERE condition is applied before GROUP BY (i.e. $group) and HAVING condition is applied after GROUP BY.<\/p>\n<p><strong>SQL Query:<\/strong><\/p>\n<pre class=\"lang:tsql decode:true\">SELECT Country,Count(*) as OrderCount\r\nFROM Orders \r\nWHERE Quantity &gt; 1\r\nGROUP BY Country\r\nHAVING Count(*) &gt; 0<\/pre>\n<p><strong>Mongo Shell Query:<\/strong><\/p>\n<pre class=\"lang:default highlight:0 decode:true\">db.Orders.aggregate(\r\n [ \r\n   {$match : {Quantity: { $gt: 1 }}}\r\n  ,{$group : {_id : \"$Country\",OrderCount: { $sum: 1 }}} \r\n  ,{$match : {OrderCount: { $gte: 1 }}}\u00a0\r\n  ,{$project : {Country: \"$_id\", _id: 0, OrderCount: 1 }}\r\n ]\r\n)<\/pre>\n<p><strong>SSIS MongoDB Source Query:<\/strong><\/p>\n<pre class=\"lang:default highlight:0 decode:true\">{Table: Orders,Method: aggregate} \r\n{ pipeline: \r\n   [\r\n      {\r\n        $group : {_id : \"$Country\",OrderCount: { $sum: 1 }}\r\n      }      \r\n   ]\r\n}<\/pre>\n<h3>Example 4 &#8211;\u00a0MongoDB Aggregation Query with $unwind (Flatten nested Array)<\/h3>\n<p>Now lets look at an example of <a href=\"https:\/\/docs.mongodb.com\/manual\/reference\/operator\/aggregation\/unwind\/\" target=\"_blank\" rel=\"noopener\">$unwind operator<\/a>. $unwind can be used to perform operations on nested array. Common things would be flatten the hierarchy and then apply group by operation. Lets find out Average Quantity of Items By Order Number<\/p>\n<p><strong>SQL Query:<\/strong><\/p>\n<pre class=\"lang:tsql decode:true\">SELECT Orders.OrderId as OrderNumber,Avg(Items.Quantity) as AvgItemQty\r\nFROM Orders \r\nJOIN Items ON Orders.OrderId = Items.OrderId\r\nGROUP BY Orders.OrderId<\/pre>\n<p><strong>Mongo Shell Query:<\/strong><\/p>\n<pre class=\"lang:default highlight:0 decode:true\">db.Orders.aggregate(\r\n [ \r\n   {$unwind : \"$Items\"}\r\n  ,{$group : {_id : \"$OrderId\",AvgItemQty: { $avg: \"$Items.Quantity\" }}} \r\n  ,{$project : {OrderNumber: \"$_id\", _id: 0, AvgItemQty: 1 }}\r\n ]\r\n)<\/pre>\n<p><strong>SSIS MongoDB Source Query:<\/strong><\/p>\n<pre class=\"lang:default highlight:0 decode:true \">{Table: Orders,Method: aggregate} \r\n{ pipeline: \r\n  [ \r\n    {$unwind : \"$Items\"}\r\n   ,{$group : {_id : \"$OrderId\",AvgItemQty: { $avg: \"$Items.Quantity\" }}} \r\n   ,{$project : {OrderNumber: \"$_id\", _id: 0, AvgItemQty: 1 }}\r\n  ]\r\n)\r\n\r\n<\/pre>\n<h2><\/h2>\n<h2>MongoDB Query with Dynamic Value \/ ISO DateTime<\/h2>\n<p>There will be a time when you have to supply Dynamic Value from Variable or supply Dynamic Date time inside your query. Dates are little different than other data types. <a href=\"https:\/\/zappysys.com\/blog\/how-to-query-mongodb-by-date-or-isodate\/\" target=\"_blank\" rel=\"noopener\">Check this article<\/a> to learn more about passing dates in MongoDB Query.<\/p>\n<p>For Dynamic Date use SSIS Variables <a href=\"https:\/\/zappysys.com\/onlinehelp\/ssis-powerpack\/scr\/ssis-format-specifiers.htm\" target=\"_blank\" rel=\"noopener\">Format Specifiers<\/a><\/p>\n<h3><strong>Example of DateTime query (Static Value)<\/strong><\/h3>\n<pre class=\"lang:default highlight:0 decode:true\">{Table: Orders,Method: aggregate} \r\n{ pipeline: \r\n   [\r\n     {$match : {OrderDate : { $gt: new ISODate(\"2010-01-01\") }}} \r\n    ,{$group : {_id : \"$Country\",OrderCount: { $sum: 1 }}} \r\n    ,{$project : {Country: \"$_id\", _id: 0, OrderCount: 1 }} ,{$sort : {Country: 1}}  \r\n   ]\r\n}<\/pre>\n<h3><strong>Example of using Dynamic Value from Variable<\/strong><\/h3>\n<p>To pass date from SSIS Variable you can use\u00a0<a href=\"https:\/\/zappysys.com\/onlinehelp\/ssis-powerpack\/scr\/ssis-format-specifiers.htm\" target=\"_blank\" rel=\"noopener\">Format Specifiers<\/a>. (e.g. <strong>yyyy-MM-dd<\/strong>\u00a0 or for full ISO date\u00a0<strong>yyyy-MM-ddTHH:mm:ssZ<\/strong>\u00a0)<\/p>\n<pre class=\"lang:default highlight:0 decode:true\">{Table: Orders,Method: aggregate} \r\n{ pipeline: \r\n   [\r\n     {$match : {OrderDate : { $gt: new ISODate(\"{{User::varOrderDate,yyyy-MM-dd}}\") }}} \r\n    ,{$group : {_id : \"$Country\",OrderCount: { $sum: 1 }}} \r\n    ,{$project : {Country: \"$_id\", _id: 0, OrderCount: 1 }} ,{$sort : {Country: 1}}  \r\n   ]\r\n}<\/pre>\n<h3><\/h3>\n<h2>Conclusion<\/h2>\n<p>MongoDB is one of the most popular NoSQL \/ Document Storage Engine. <a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/\" target=\"_blank\" rel=\"noopener\">SSIS PowerPack<\/a> make super easy and efficient to read\/write data in MongoDB without any coding effort. <a href=\"https:\/\/zappysys.com\/products\/ssis-powerpack\/download\/\" target=\"_blank\" rel=\"noopener\">Download SSIS PowerPack<\/a>\u00a0to try out MongoDB features along with 70+ other components .<\/p>\n<p><strong>Search terms:<\/strong><br \/>\nmongodb group by count<br \/>\nmongodb group by multiple fields<br \/>\nmongodb group by date<br \/>\nmongodb group example<br \/>\nmongodb group sort<br \/>\nmongodb group by month<br \/>\nmongodb group by having<br \/>\nmongodb group distinct<br \/>\nmongodb group by sum<br \/>\nmongodb aggregate<br \/>\nmongodb aggregate sort<br \/>\nmongodb aggregate unwind<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction In this post you will learn how to write flexible MongoDB Aggregation Queries in SSIS (i.e. Group By Query) using SSIS MongoDB Source Component. MongoDB provides powerful Aggregation Pipeline Engine which is conceptually same as writing Group By queries in traditional SQL world but its not exactly same.\u00a0SSIS MongoDB Source returns nested MongoDB JSON [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":2115,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[46],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\r\n<title>How to write MongoDB Aggregation Queries in SSIS (Group By) | ZappySys Blog<\/title>\r\n<meta name=\"description\" content=\"Learn how to aggregate MongoDB data using group by query in SSIS. Perform SUM, COUNT, MAX, MIN with group by and having clause similar in SQL query language\" \/>\r\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\r\n<link rel=\"canonical\" href=\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/\" \/>\r\n<meta property=\"og:locale\" content=\"en_US\" \/>\r\n<meta property=\"og:type\" content=\"article\" \/>\r\n<meta property=\"og:title\" content=\"How to write MongoDB Aggregation Queries in SSIS (Group By) | ZappySys Blog\" \/>\r\n<meta property=\"og:description\" content=\"Learn how to aggregate MongoDB data using group by query in SSIS. Perform SUM, COUNT, MAX, MIN with group by and having clause similar in SQL query language\" \/>\r\n<meta property=\"og:url\" content=\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/\" \/>\r\n<meta property=\"og:site_name\" content=\"ZappySys Blog\" \/>\r\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/ZappySys\/\" \/>\r\n<meta property=\"article:published_time\" content=\"2016-08-17T15:52:17+00:00\" \/>\r\n<meta property=\"article:modified_time\" content=\"2024-07-10T21:06:01+00:00\" \/>\r\n<meta property=\"og:image\" content=\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2017\/08\/mongodb-logo.png\" \/>\r\n\t<meta property=\"og:image:width\" content=\"88\" \/>\r\n\t<meta property=\"og:image:height\" content=\"88\" \/>\r\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\r\n<meta name=\"author\" content=\"ZappySys\" \/>\r\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\r\n<meta name=\"twitter:creator\" content=\"@https:\/\/twitter.com\/zappysys\/\" \/>\r\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"ZappySys\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\r\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/\",\"url\":\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/\",\"name\":\"How to write MongoDB Aggregation Queries in SSIS (Group By) | ZappySys Blog\",\"isPartOf\":{\"@id\":\"https:\/\/zappysys.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2017\/08\/mongodb-logo.png\",\"datePublished\":\"2016-08-17T15:52:17+00:00\",\"dateModified\":\"2024-07-10T21:06:01+00:00\",\"author\":{\"@id\":\"https:\/\/zappysys.com\/blog\/#\/schema\/person\/2756c237457fbc95d82cb38962f81f82\"},\"description\":\"Learn how to aggregate MongoDB data using group by query in SSIS. Perform SUM, COUNT, MAX, MIN with group by and having clause similar in SQL query language\",\"breadcrumb\":{\"@id\":\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#primaryimage\",\"url\":\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2017\/08\/mongodb-logo.png\",\"contentUrl\":\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2017\/08\/mongodb-logo.png\",\"width\":88,\"height\":88},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/zappysys.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to write MongoDB Aggregation Queries in SSIS (Group By)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/zappysys.com\/blog\/#website\",\"url\":\"https:\/\/zappysys.com\/blog\/\",\"name\":\"ZappySys Blog\",\"description\":\"SSIS \/ ODBC Drivers \/ API Connectors for JSON, XML, Azure, Amazon AWS, Salesforce, MongoDB and more\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/zappysys.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/zappysys.com\/blog\/#\/schema\/person\/2756c237457fbc95d82cb38962f81f82\",\"name\":\"ZappySys\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/zappysys.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/5c9be148088ba9b8af8e955c5f7c22b5?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/5c9be148088ba9b8af8e955c5f7c22b5?s=96&d=mm&r=g\",\"caption\":\"ZappySys\"},\"sameAs\":[\"http:\/\/www.zappysys.com\/\",\"https:\/\/www.facebook.com\/ZappySys\/\",\"https:\/\/twitter.com\/https:\/\/twitter.com\/zappysys\/\"],\"url\":\"https:\/\/zappysys.com\/blog\/author\/admin\/\"}]}<\/script>\r\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How to write MongoDB Aggregation Queries in SSIS (Group By) | ZappySys Blog","description":"Learn how to aggregate MongoDB data using group by query in SSIS. Perform SUM, COUNT, MAX, MIN with group by and having clause similar in SQL query language","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/","og_locale":"en_US","og_type":"article","og_title":"How to write MongoDB Aggregation Queries in SSIS (Group By) | ZappySys Blog","og_description":"Learn how to aggregate MongoDB data using group by query in SSIS. Perform SUM, COUNT, MAX, MIN with group by and having clause similar in SQL query language","og_url":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/","og_site_name":"ZappySys Blog","article_author":"https:\/\/www.facebook.com\/ZappySys\/","article_published_time":"2016-08-17T15:52:17+00:00","article_modified_time":"2024-07-10T21:06:01+00:00","og_image":[{"width":88,"height":88,"url":"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2017\/08\/mongodb-logo.png","type":"image\/png"}],"author":"ZappySys","twitter_card":"summary_large_image","twitter_creator":"@https:\/\/twitter.com\/zappysys\/","twitter_misc":{"Written by":"ZappySys","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/","url":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/","name":"How to write MongoDB Aggregation Queries in SSIS (Group By) | ZappySys Blog","isPartOf":{"@id":"https:\/\/zappysys.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#primaryimage"},"image":{"@id":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#primaryimage"},"thumbnailUrl":"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2017\/08\/mongodb-logo.png","datePublished":"2016-08-17T15:52:17+00:00","dateModified":"2024-07-10T21:06:01+00:00","author":{"@id":"https:\/\/zappysys.com\/blog\/#\/schema\/person\/2756c237457fbc95d82cb38962f81f82"},"description":"Learn how to aggregate MongoDB data using group by query in SSIS. Perform SUM, COUNT, MAX, MIN with group by and having clause similar in SQL query language","breadcrumb":{"@id":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#primaryimage","url":"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2017\/08\/mongodb-logo.png","contentUrl":"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2017\/08\/mongodb-logo.png","width":88,"height":88},{"@type":"BreadcrumbList","@id":"https:\/\/zappysys.com\/blog\/aggregate-mongodb-data-using-group-by-query-in-ssis\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/zappysys.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How to write MongoDB Aggregation Queries in SSIS (Group By)"}]},{"@type":"WebSite","@id":"https:\/\/zappysys.com\/blog\/#website","url":"https:\/\/zappysys.com\/blog\/","name":"ZappySys Blog","description":"SSIS \/ ODBC Drivers \/ API Connectors for JSON, XML, Azure, Amazon AWS, Salesforce, MongoDB and more","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/zappysys.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/zappysys.com\/blog\/#\/schema\/person\/2756c237457fbc95d82cb38962f81f82","name":"ZappySys","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/zappysys.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/5c9be148088ba9b8af8e955c5f7c22b5?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/5c9be148088ba9b8af8e955c5f7c22b5?s=96&d=mm&r=g","caption":"ZappySys"},"sameAs":["http:\/\/www.zappysys.com\/","https:\/\/www.facebook.com\/ZappySys\/","https:\/\/twitter.com\/https:\/\/twitter.com\/zappysys\/"],"url":"https:\/\/zappysys.com\/blog\/author\/admin\/"}]}},"_links":{"self":[{"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/posts\/736"}],"collection":[{"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/comments?post=736"}],"version-history":[{"count":18,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/posts\/736\/revisions"}],"predecessor-version":[{"id":11126,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/posts\/736\/revisions\/11126"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/media\/2115"}],"wp:attachment":[{"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/media?parent=736"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/categories?post=736"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/tags?post=736"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}