{"id":9918,"date":"2023-03-27T11:49:52","date_gmt":"2023-03-27T11:49:52","guid":{"rendered":"https:\/\/zappysys.com\/blog\/?p=9918"},"modified":"2024-06-27T13:58:08","modified_gmt":"2024-06-27T13:58:08","slug":"how-to-download-images-from-a-web-page-using-ssis","status":"publish","type":"post","link":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/","title":{"rendered":"How to download images from a web page using SSIS"},"content":{"rendered":"

Introduction<\/strong><\/h2>\n

The article shows how to download images from a web page using SSIS. Sometimes we need to download the images from a web page. In this post, we will show how to do this.<\/p>\n

Prerequisites<\/strong><\/h2>\n

Before we perform steps listed in this article, you will need to make sure following prerequisites are met:<\/p>\n

    \n
  1. SSIS designer installed. Sometimes it is referred as BIDS or SSDT (download it from Microsoft site<\/a>).<\/li>\n
  2. Basic knowledge of SSIS package development using Microsoft SQL Server Integration Services.<\/li>\n
  3. Make sure ZappySys SSIS PowerPack<\/a> is installed (download it<\/a>).<\/li>\n
  4. Optional (If you want to Deploy and Schedule ) – Deploy and Schedule SSIS Packages<\/a><\/li>\n<\/ol>\n

    Steps-by-step process to download images from HTML using SSIS<\/strong><\/h2>\n

    Use REST API task to get the HTML body<\/h3>\n

    1. Drag and drop the REST API Task from SSIS toolbox, select the html page you want and save it in a variable.<\/p>\n

    \"\"<\/a>

    Select the page you want to get the images<\/p><\/div>\n

    2. Go to Response Settings.\u00a0<\/strong>Check the option Save the response content. In Save Mode,\u00a0<\/strong>select Save to File. <\/strong>In the option Enter File Path<\/strong>, write the path for the html file.<\/p>\n

    \"\"<\/a>

    Save the page in a file<\/p><\/div>\n

    Parse the image with Regular Expression parser task<\/h3>\n

    3. From the SSIS toolbox drag and drop Regular Expression Parser Task on the Control flow designer surface.<\/p>\n

    4. The next step is to save the source path from the images. You need to use Regex and here are two example you can use, in this page Regex101<\/a> you can check more details about the expressions we are using:<\/p>\n

    Expression 1: <img.*?src=”(.*?)”{{*}}<\/span>
    \nExpression 2: src=”([a-z\\-_0-9\\\/\\:\\.]*\\.(png|jpg|jpeg|gif|png))”{{*}}<\/span><\/p>\n

    \"download<\/a>

    Regex expression to get the image code from the page<\/p><\/div>\n

    Read the image source with CSV source in order to\u00a0download images from a Web page<\/h3>\n

    5. Now, Drag and Drop SSIS\u00a0Data Flow Task<\/b>\u00a0from SSIS Toolbox.<\/p>\n

    <\/p>\n

    6. Double click on the DataFlow task to see DataFlow designer surface.<\/p>\n

    7. From the SSIS toolbox drag and drop CSV source and insert the variable you are using from the previous task<\/p>\n

    \"CSV<\/a>

    Use CSV source to read the variable<\/p><\/div>\n

    Get the image name, image full path and the destination folder to download images from a Web Page<\/h3>\n

    8. From the SSIS toolbox drag and drop\u00a0Derived Column transform to remove the HTML code and get the image name.<\/p>\n

    Expression for the Image name:<\/p>\n

    (DT_WSTR,1000)REVERSE(LEFT(REVERSE(REPLACE(REPLACE(Column1,\"src=\\\"\",\"\"),\"\\\"\",\"\")),(FINDSTRING(REVERSE(REPLACE(REPLACE(Column1,\"src=\\\"\",\"\"),\"\\\"\",\"\")),\"\/\",1) - 1)))<\/pre>\n
    Remove the HTML code:<\/p>\n
    REPLACE(REPLACE(Column1,\"src=\\\"\",\"\"),\"\\\"\",\"\")<\/pre>\n
    \"Image<\/a>
    Add a new column and remove HTML code<\/p><\/div>\n
    9. Now we will drag and drop another Derived Column, this one is for the file path for the images and validate if the image URL is valid. If the image URL has \u201chttp\u201d in the stringm then it is valid otherwise you need to add the rest of the URL. Here is our example for the filePath and validate the URL image.\u00a0If everything is OK, we can send the image URL.<\/p>\n
    File path: (DT_WSTR,2500)@[User::vDestinationFolder] + ImageName\r\n\r\nValidate image URL: ((FINDSTRING(Column1,\"http\",1) > 0) ? Column1 : (DT_WSTR,2500)(\"https:\/\/www.urlexample\/image\/\" + Column1))\r\n\r\nNote: @[User::vDestinationFolder] is a variable we used for the local path<\/pre>\n
    \"Expression<\/a>
    Add a new column and verify the HTML path<\/p><\/div>\n
    Send the request and save the image In the folder TO\u00a0download images from a Web Page<\/h3>\n
    10. Now we will drag and drop a web API destination and select the column in the input column to URL.\u00a0If the request is right, we can save the image in the local file.<\/p>\n
    \"Web<\/a>
    Send a request for all images you get<\/p><\/div>\n
    11. Now\u00a0drag and drop an Export Column transformation and select the ResponseText from the request and the file path.<\/p>\n
    \"download<\/a>
    Select the image response and the local file path<\/p><\/div>\n
    12. Finally, we use a trash destination to close the flow.<\/p>\n
    \"download<\/a>
    The final result<\/p><\/div>\n
    Conclusion<\/h2>\n
    If everything is OK, you will be able to download the images from your HTML page. To do that, we read the list of URLs from a list. Then we get the name of the images using expressions. Then we add a path for each image to store them. Finally, we save the images.<\/p>\n","protected":false},"excerpt":{"rendered":"
    Introduction The article shows how to download images from a web page using SSIS. Sometimes we need to download the images from a web page. In this post, we will show how to do this. Prerequisites Before we perform steps listed in this article, you will need to make sure following prerequisites are met: SSIS […]<\/p>\n","protected":false},"author":5,"featured_media":9958,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[176,447],"tags":[531,609,3],"yoast_head":"\r\nHow to download images from a web page using SSIS | ZappySys Blog<\/title>\r\n<meta name=\"description\" content=\"In this article we will see how to download images from a web page using SQL Server Integration Services. We will show step by step how to do it.\" \/>\r\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\r\n<link rel=\"canonical\" href=\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/\" \/>\r\n<meta property=\"og:locale\" content=\"en_US\" \/>\r\n<meta property=\"og:type\" content=\"article\" \/>\r\n<meta property=\"og:title\" content=\"How to download images from a web page using SSIS | ZappySys Blog\" \/>\r\n<meta property=\"og:description\" content=\"In this article we will see how to download images from a web page using SQL Server Integration Services. We will show step by step how to do it.\" \/>\r\n<meta property=\"og:url\" content=\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/\" \/>\r\n<meta property=\"og:site_name\" content=\"ZappySys Blog\" \/>\r\n<meta property=\"article:published_time\" content=\"2023-03-27T11:49:52+00:00\" \/>\r\n<meta property=\"article:modified_time\" content=\"2024-06-27T13:58:08+00:00\" \/>\r\n<meta property=\"og:image\" content=\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2023\/02\/RAT1.png\" \/>\r\n\t<meta property=\"og:image:width\" content=\"836\" \/>\r\n\t<meta property=\"og:image:height\" content=\"713\" \/>\r\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\r\n<meta name=\"author\" content=\"ZappySys Team\" \/>\r\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\r\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"ZappySys Team\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\r\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/\",\"url\":\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/\",\"name\":\"How to download images from a web page using SSIS | ZappySys Blog\",\"isPartOf\":{\"@id\":\"https:\/\/zappysys.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2023\/02\/RAT1.png\",\"datePublished\":\"2023-03-27T11:49:52+00:00\",\"dateModified\":\"2024-06-27T13:58:08+00:00\",\"author\":{\"@id\":\"https:\/\/zappysys.com\/blog\/#\/schema\/person\/91b041e2dcf7ece5f068893c1a68ac6e\"},\"description\":\"In this article we will see how to download images from a web page using SQL Server Integration Services. We will show step by step how to do it.\",\"breadcrumb\":{\"@id\":\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#primaryimage\",\"url\":\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2023\/02\/RAT1.png\",\"contentUrl\":\"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2023\/02\/RAT1.png\",\"width\":836,\"height\":713,\"caption\":\"Select the page you want to get the images\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/zappysys.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How to download images from a web page using SSIS\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/zappysys.com\/blog\/#website\",\"url\":\"https:\/\/zappysys.com\/blog\/\",\"name\":\"ZappySys Blog\",\"description\":\"SSIS \/ ODBC Drivers \/ API Connectors for JSON, XML, Azure, Amazon AWS, Salesforce, MongoDB and more\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/zappysys.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/zappysys.com\/blog\/#\/schema\/person\/91b041e2dcf7ece5f068893c1a68ac6e\",\"name\":\"ZappySys Team\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/zappysys.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/746bec9c9d27f1b90bb181aa516ee234?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/746bec9c9d27f1b90bb181aa516ee234?s=96&d=mm&r=g\",\"caption\":\"ZappySys Team\"},\"sameAs\":[\"https:\/\/zappysys.com\"],\"url\":\"https:\/\/zappysys.com\/blog\/author\/dcalbimonte\/\"}]}<\/script>\r\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How to download images from a web page using SSIS | ZappySys Blog","description":"In this article we will see how to download images from a web page using SQL Server Integration Services. We will show step by step how to do it.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/","og_locale":"en_US","og_type":"article","og_title":"How to download images from a web page using SSIS | ZappySys Blog","og_description":"In this article we will see how to download images from a web page using SQL Server Integration Services. We will show step by step how to do it.","og_url":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/","og_site_name":"ZappySys Blog","article_published_time":"2023-03-27T11:49:52+00:00","article_modified_time":"2024-06-27T13:58:08+00:00","og_image":[{"width":836,"height":713,"url":"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2023\/02\/RAT1.png","type":"image\/png"}],"author":"ZappySys Team","twitter_card":"summary_large_image","twitter_misc":{"Written by":"ZappySys Team","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/","url":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/","name":"How to download images from a web page using SSIS | ZappySys Blog","isPartOf":{"@id":"https:\/\/zappysys.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#primaryimage"},"image":{"@id":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#primaryimage"},"thumbnailUrl":"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2023\/02\/RAT1.png","datePublished":"2023-03-27T11:49:52+00:00","dateModified":"2024-06-27T13:58:08+00:00","author":{"@id":"https:\/\/zappysys.com\/blog\/#\/schema\/person\/91b041e2dcf7ece5f068893c1a68ac6e"},"description":"In this article we will see how to download images from a web page using SQL Server Integration Services. We will show step by step how to do it.","breadcrumb":{"@id":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#primaryimage","url":"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2023\/02\/RAT1.png","contentUrl":"https:\/\/zappysys.com\/blog\/wp-content\/uploads\/2023\/02\/RAT1.png","width":836,"height":713,"caption":"Select the page you want to get the images"},{"@type":"BreadcrumbList","@id":"https:\/\/zappysys.com\/blog\/how-to-download-images-from-a-web-page-using-ssis\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/zappysys.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How to download images from a web page using SSIS"}]},{"@type":"WebSite","@id":"https:\/\/zappysys.com\/blog\/#website","url":"https:\/\/zappysys.com\/blog\/","name":"ZappySys Blog","description":"SSIS \/ ODBC Drivers \/ API Connectors for JSON, XML, Azure, Amazon AWS, Salesforce, MongoDB and more","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/zappysys.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/zappysys.com\/blog\/#\/schema\/person\/91b041e2dcf7ece5f068893c1a68ac6e","name":"ZappySys Team","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/zappysys.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/746bec9c9d27f1b90bb181aa516ee234?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/746bec9c9d27f1b90bb181aa516ee234?s=96&d=mm&r=g","caption":"ZappySys Team"},"sameAs":["https:\/\/zappysys.com"],"url":"https:\/\/zappysys.com\/blog\/author\/dcalbimonte\/"}]}},"_links":{"self":[{"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/posts\/9918"}],"collection":[{"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/comments?post=9918"}],"version-history":[{"count":25,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/posts\/9918\/revisions"}],"predecessor-version":[{"id":11488,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/posts\/9918\/revisions\/11488"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/media\/9958"}],"wp:attachment":[{"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/media?parent=9918"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/categories?post=9918"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/zappysys.com\/blog\/wp-json\/wp\/v2\/tags?post=9918"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}