update readme

- Fix monthly price calculation

.gitignore

@@ -280,4 +280,6 @@ Network Trash Folder
 Temporary Items
 .apdisk
 
+clouds.yaml
+
 # End of https://www.toptal.com/developers/gitignore/api/macos

CHANGELOG.txt

@@ -1,3 +1,12 @@
+Next version
+--------------------------
+- Feature: Project balance
+- Feature: Prometeus metric export
+- Feature: Unpaid Invoice handling
+- Fix: Money maximum digit
+- Fix: Fix monthly price calculation that use UTC timezone
+- Add setup_cron.sh script to easily setup yuyu cronjob
+
 v1.0.1
 ---------------------------
 - Fix: Support decimal in price (database migration needed)

README.md

@@ -1,17 +1,21 @@
 # Yuyu
 
-Yuyu provide ability to manage openstack billing by listening to every openstack event. Yuyu is a required component to use Yuyu Dashboard. There are 3 main component in Yuyu: API, Cron, Event Monitor 
+Yuyu provide ability to manage openstack billing by listening to every openstack event. Yuyu is a required component to use Yuyu Dashboard. There are 3 main component in Yuyu: API, Cron, Event Monitor
 
 ## Yuyu API
+
 Main component to communicate with Yuyu Dashboard.
 
 ## Yuyu Cron
+
 Provide invoice calculation and rolling capabilities that needed to run every month.
 
 ## Yuyu Event Monitor
+
 Monitor event from openstack to calculate billing spent.
 
 # System Requirement
+
 - Python 3
 - Openstack
 - Virtualenv
@@ -20,6 +24,7 @@ Monitor event from openstack to calculate billing spent.
 # Pre-Installation
 
 ### Virtualenv
+
 Make sure you installed virtualenv before installing Yuyu
 
 ```bash
@@ -31,13 +36,16 @@ pip3 install virtualenv
 Billing is a time sensitive application, please make sure you set a correct time and timezone on you machine.
 
 ### Openstack Service Notification
+
 You need to enable notification for this openstack service:
+
 - Nova (nova.conf)
 - Cinder (cinder.conf)
 - Neutron (neutron.conf)
 - Keystone (keystone.conf)
 
 ### Nova
+
 Add configuration below on `[oslo_messaging_notifications]`
 
 ```
@@ -62,7 +70,9 @@ topics = notifications
 ```
 
 ### Kolla Note
-If you using Kolla, please add configuration above on all service container. For example on Nova you should put the config on `nova-api`, `nova-scheduler`, etc.
+
+If you using Kolla, please add configuration above on all service container. For example on Nova you should put the
+config on `nova-api`, `nova-scheduler`, etc.
 
 # Installation
 
@@ -75,6 +85,7 @@ cd yuyu
 ```
 
 Then create virtualenv and activate it
+
 ```bash
 virtualenv env --python=python3.8
 source env/bin/activate
@@ -87,7 +98,8 @@ Then create a configuration file, just copy from sample file and modify as your
 cp yuyu/local_settings.py.sample yuyu/local_settings.py
 ```
 
-Please read [Local Setting Configuration](#local-setting-configuration) to get to know about what configuration you should change.
+Please read [Local Setting Configuration](#local-setting-configuration) to get to know about what configuration you
+should change.
 
 Then run the database migration
 
@@ -104,23 +116,27 @@ python manage.py createsuperuser
 ## Local Setting Configuration
 
 ### YUYU_NOTIFICATION_URL (required)
+
 A Messaging Queue URL that used by Openstack, usually it is a RabbitMQ URL.
 
-Example: 
+Example:
+
 ```
 YUYU_NOTIFICATION_URL = "rabbit://openstack:password@127.0.0.1:5672/"
 ```
 
 ### YUYU_NOTIFICATION_TOPICS (required)
+
 A list of topic notification topic that is configured on each openstack service
 
-Example: 
+Example:
+
 ```
 YUYU_NOTIFICATION_TOPICS = ["notifications"]
 ```
 
-
 ### DATABASE
+
 By default, it will use Sqlite. If you want to change it to other database please refer to Django Setting documentation.
 
 - https://docs.djangoproject.com/en/3.2/ref/settings/#databases
@@ -137,12 +153,13 @@ To install Yuyu API, you need to execute this command.
 This will install `yuyu_api` service
 
 To start the service use this command
+
 ```bash
 systemctl enable yuyu_api
 systemctl start yuyu_api
 ```
 
-An API server will be open on port `8182`. 
+An API server will be open on port `8182`.
 
 ## Event Monitor Installation
 
@@ -152,10 +169,10 @@ To install Yuyu API, you need to execute this command.
 ./bin/setup_event_monitor.sh
 ```
 
-
 This will install `yuyu_event_monitor` service
 
 To start the service use this command
+
 ```bash
 systemctl enable yuyu_event_monitor
 systemctl start yuyu_event_monitor
@@ -163,20 +180,15 @@ systemctl start yuyu_event_monitor
 
 ## Cron Installation
 
-There is a cronjob that needed to be run every month on 00:01 AM. This cronjob will finish all in progress invoice and start new invoice for the next month.
+There is a cronjob that needed to be run every day and every month. The cronjob is used to process the invoice and handling updaid invoice.
 
-To install it, you can use `crontab -e`.
+To install the cronjob, you need to execute this command
 
-Put this expression on the crontab
+```bash
-
+./bin/setup_cron.sh
-```
-1 0 1 * * $yuyu_dir/bin/process_invoice.sh 
 ```
 
-Replace $yuyu_dir with the directory of where yuyu is located. Example
+You can make sure the cron is installed by checking `crontab -e`.
-```
-1 0 1 * * /var/yuyu/bin/process_invoice.sh
-```
 
 # Updating Yuyu
 
@@ -192,7 +204,7 @@ Activate the virtualenv.
 source env/bin/activate
 ```
 
-Change the setting if needed. 
+Change the setting if needed.
 
 ```bash
 nano yuyu/local_settings.py
@@ -215,4 +227,219 @@ Restart all the service
 ```bash
 systemctl restart yuyu_api
 systemctl restart yuyu_event_monitor
+```
+
+# Other Feature
+
+Yuyu also have other feature that need to be setup to use.
+
+## Unpaid Invoice Handling
+
+Unpaid invoice handling can run a task on a specific day after invoice is issued and not yet paid.
+
+For example, you can send email to remind customer to pay the invoice or delete all instance on a specific day.
+
+Available action for unpaid invoice handling consist of:
+
+- send_message : Sending a message to a customer
+- pause_instance : Will pause compute instance
+- suspend_instance : Will suspend compute instance
+- stop_instance : Will stop compute instance
+- delete_instance : Will delete compute, image, router, snapshot or volume
+
+
+### Enabling Unpaid Invoice Handling
+
+1. Download `clouds.yaml` configuration from Openstack API Dashboard
+2. Make sure you also set the password in `clouds.yaml`. Example:
+   ```yaml
+   clouds:
+     openstack:
+       auth:
+         auth_url: http://172.10.10.150:5000
+         username: "admin"
+         password: "your_password"
+         project_id: 0000000000000000000000000
+         project_name: "admin"
+         user_domain_name: "Default"
+       region_name: "ID"
+       interface: "public"
+       identity_api_version: 3
+   ```
+3. Put `clouds.yaml` in one of the following directory.
+    - Current Yuyu Directory
+    - ~/.config/openstack
+    - /etc/openstack
+4. Setup the configuration in `local_settings.py`. See **Configuration** for detail.
+
+   Example config can follow, you can put any number of config or remove it completely to disable it.
+   ```python
+   CLOUD_CONFIG_NAME = "openstack"
+   UNPAID_INVOICE_HANDLER_CONFIG = [
+       {
+           "day": 5,
+           "action": "send_message",
+           "message_title": "Your invoice has been expired. Please pay now!",
+           "message_short_description": "Your invoice has been expired. Please pay now!",
+           "message_content": "Your invoice has been expired. Please pay now!",
+       },
+       {
+           "day": 10,
+           "action": "stop_instance",
+       },
+       {
+           "day": 10,
+           "action": "send_message",
+           "message_title": "Your compute instance will be stopped",
+           "message_short_description": "Your compute instance will be stopped",
+           "message_content": "Your compute instance will be stopped because you have unpaid invoice",
+       },
+       {
+           "day": 15,
+           "action": "send_message",
+           "message_title": "All of your instance has been deleted",
+           "message_short_description": "All of your instance has been deleted",
+           "message_content": "All of your instance has been deleted because you have unpaid invoice",
+       },
+       {
+           "day": 15,
+           "action": "delete_instance",
+       },
+   ]
+   ```
+5. Setup `cronjob` to run the action every day
+   
+   To install it, you need to execute this command
+
+   ```bash
+   ./bin/setup_cron.sh
+   ```
+
+   You can make sure the cron is installed by checking `crontab -e` and make sure `handle_unpaid_invoice.sh` is present
+
+6. Check the connection to openstack with
+   ```
+   ./bin/check_openstack_connection.sh
+   ```
+   Make sure it doesn't return error.
+7. Done
+
+### Configuration
+
+To use Unpaid Invoice Handling you need to set up the following variable in `local_settings.py`
+
+- CLOUD_CONFIG_NAME
+- UNPAID_INVOICE_HANDLER_CONFIG
+
+#### CLOUD_CONFIG_NAME
+
+CLOUD_CONFIG_NAME Is configuration in `clouds.yaml` that you want to use to connect to openstack.
+
+Example: Your `clouds.yaml` is
+
+```yaml
+clouds:
+  openstack:
+    auth:
+      auth_url: http://172.10.10.150:5000
+      username: "admin"
+      password: "your_password"
+      project_id: 0000000000000000000000000
+      project_name: "admin"
+      user_domain_name: "Default"
+    region_name: "ID"
+    interface: "public"
+    identity_api_version: 3
+```
+
+You can put `openstack` as `CLOUD_CONFIG_NAME`
+
+So it will be
+
+```python
+CLOUD_CONFIG_NAME = "openstack"
+```
+
+### UNPAID_INVOICE_HANDLER_CONFIG
+
+UNPAID_INVOICE_HANDLER_CONFIG Is configuration for an action that will be run on a particular day after invoice is
+issued and still unpaid
+
+The available action that can be used is
+
+- send_message : Sending a message to a customer
+- stop_instance : Will stop, compute instance
+- delete_instance : Will delete compute, image, router, snapshot or volume
+
+UNPAID_INVOICE_HANDLER_CONFIG is list of dictionary, you can add as many config as you want to the list that will be run
+on a particular day.
+
+The format for config dictionary is
+
+```python
+{
+    "day": 0,
+    "action": "the_action",
+}
+```
+
+The most important part is `day` and `action`.
+
+- `day`: The day of the action that will be run. For example if you put `5` it will run in 5 day after invoice is issued
+  and still unpaid
+- `action`: The action that you want to run. It can be `send_message`/`stop_instance`/`delete_instance`.
+
+If you use `send_message` action, you need to add additional config.
+
+- `message_title` : The title or subject of the message
+- `message_short_description` : The short description of message
+- `message_content`: The content of the message
+
+Example:
+
+```python
+{
+    "day": 0,
+    "action": "send_message",
+    "message_title": "Title",
+    "message_short_description": "Short Description",
+    "message_content": "The Content",
+}
+```
+
+This is example config that you can use as a reference
+
+```python
+CLOUD_CONFIG_NAME = "openstack"
+UNPAID_INVOICE_HANDLER_CONFIG = [
+    {
+        "day": 5,
+        "action": "send_message",
+        "message_title": "Your invoice has been expired. Please pay now!",
+        "message_short_description": "Your invoice has been expired. Please pay now!",
+        "message_content": "Your invoice has been expired. Please pay now!",
+    },
+    {
+        "day": 10,
+        "action": "stop_instance",
+    },
+    {
+        "day": 10,
+        "action": "send_message",
+        "message_title": "Your compute instance will be stopped",
+        "message_short_description": "Your compute instance will be stopped",
+        "message_content": "Your compute instance will be stopped because you have unpaid invoice",
+    },
+    {
+        "day": 15,
+        "action": "send_message",
+        "message_title": "All of your instance has been deleted",
+        "message_short_description": "All of your instance has been deleted",
+        "message_content": "All of your instance has been deleted because you have unpaid invoice",
+    },
+    {
+        "day": 15,
+        "action": "delete_instance",
+    },
+]
 ```

api/views.py

@@ -26,6 +26,7 @@ from core.component.component import INVOICE_COMPONENT_MODEL
 from core.exception import PriceNotFound
 from core.models import Invoice, BillingProject, Notification, Balance, InvoiceInstance
 from core.notification import send_notification_from_template
+from core.utils.date_utils import current_localtime
 from core.utils.dynamic_setting import (
     get_dynamic_settings,
     get_dynamic_setting,
@@ -226,7 +227,7 @@ class InvoiceViewSet(viewsets.ModelViewSet):
         ).all()
         for active_invoice in active_invoices:
             self._close_active_invoice(
-                active_invoice, timezone.now(), get_dynamic_setting(INVOICE_TAX)
+                active_invoice, current_localtime(), get_dynamic_setting(INVOICE_TAX)
             )
 
         return Response({"status": "success"})
@@ -249,10 +250,30 @@ class InvoiceViewSet(viewsets.ModelViewSet):
         active_invoice.close(close_date, tax_percentage)
 
     @action(detail=False, methods=["POST"])
+    @transaction.atomic
     def reset_billing(self, request):
-        self.handle_reset_billing()
+        set_dynamic_setting(BILLING_ENABLED, False)
+
+        BillingProject.objects.all().delete()
+        for name, handler in component.INVOICE_HANDLER.items():
+            handler.delete()
+
+        for name, model in component.PRICE_MODEL.items():
+            model.objects.all().delete()
         return Response({"status": "success"})
 
+    @action(detail=False, methods=["POST"])
+    @transaction.atomic
+    def reset_transaction_data(self, request):
+        set_dynamic_setting(BILLING_ENABLED, False)
+
+        BillingProject.objects.all().delete()
+        for name, handler in component.INVOICE_HANDLER.items():
+            handler.delete()
+
+        return Response({"status": "success"})
+
+    
     @transaction.atomic
     def handle_init_billing(self, data):
         set_dynamic_setting(BILLING_ENABLED, True)
@@ -260,11 +281,11 @@ class InvoiceViewSet(viewsets.ModelViewSet):
         projects = {}
         invoices = {}
 
-        date_today = timezone.now()
+        date_today = current_localtime()
         month_first_day = date_today.replace(
             day=1, hour=0, minute=0, second=0, microsecond=0
         )
-
+        
         for name, handler in component.INVOICE_HANDLER.items():
             payloads = data[name]
 
@@ -294,16 +315,6 @@ class InvoiceViewSet(viewsets.ModelViewSet):
                 del payload["tenant_id"]
                 handler.create(payload, fallback_price=True)
 
-    @transaction.atomic
-    def handle_reset_billing(self):
-        set_dynamic_setting(BILLING_ENABLED, False)
-
-        BillingProject.objects.all().delete()
-        for name, handler in component.INVOICE_HANDLER.items():
-            handler.delete()
-
-        for name, model in component.PRICE_MODEL.items():
-            model.objects.all().delete()
 
     @action(detail=True)
     def finish(self, request, pk):

bin/check_openstack_connection.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+cd $SCRIPT_DIR || exit
+cd ..
+./env/bin/python manage.py check_openstack_connection

bin/handle_unpaid_invoice.sh

@@ -0,0 +1,6 @@
+#!/bin/bash
+
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+cd $SCRIPT_DIR || exit
+cd ..
+./env/bin/python manage.py handle_unpaid_invoice

bin/setup_cron.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+# Get the directory of the current script
+current_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# Define the cron jobs
+cron_job1="1 0 1 * * $current_dir/process_invoice.sh"
+cron_job2="5 0 * * * $current_dir/handle_unpaid_invoice.sh"
+
+# Define the identifiers
+identifier1="process_invoice"
+identifier2="handle_unpaid_invoice"
+
+# Check if the cron jobs already exist
+if ! crontab -l | grep -q "$identifier1"; then
+    # Add the cron job if it doesn't exist
+    (crontab -l; echo "$cron_job1") | crontab -
+fi
+
+if ! crontab -l | grep -q "$identifier2"; then
+    # Add the cron job if it doesn't exist
+    (crontab -l; echo "$cron_job2") | crontab -
+fi

bin/yuyu_test.sh

@@ -0,0 +1,52 @@
+#!/bin/bash
+
+case $1 in
+  current_time)
+    date +"Current Time: %Y-%m-%d %H:%M:%S %Z"
+    ;;
+  
+  end_of_month)
+    current_month=$(date +%m)
+    current_year=$(date +%Y)
+    last_day=$(cal $current_month $current_year | awk 'NF {DAYS = $NF} END {print DAYS}')
+    end_of_month=$(date -d "$current_year-$current_month-$last_day 23:59:00" +"%Y-%m-%d %H:%M:%S %Z")
+    sudo systemctl stop systemd-timesyncd
+    sudo date -s "$end_of_month"
+    date +"Time set to end of the month: %Y-%m-%d %H:%M:%S %Z"
+    echo "Restarting cron"
+    sudo systemctl restart cron
+    ;;
+
+  end_of_day)
+    days_offset=$2
+    end_of_day=$(date -d "+$days_offset days 23:59:00" +"%Y-%m-%d %H:%M:%S %Z")
+    sudo systemctl stop systemd-timesyncd
+    sudo date -s "$end_of_day"
+    date +"Time set to end of the day: %Y-%m-%d %H:%M:%S %Z"
+    echo "Restarting cron"
+    sudo systemctl restart cron
+    ;;
+
+  add_minutes)
+    minutes_offset=$2
+    end_of_day=$(date -d "+$minutes_offset minutes" +"%Y-%m-%d %H:%M:%S %Z")
+    sudo systemctl stop systemd-timesyncd
+    sudo date -s "$end_of_day"
+    date +"Time set to: %Y-%m-%d %H:%M:%S %Z"
+    echo "Restarting cron"
+    sudo systemctl restart cron
+    ;;
+
+  reset_time)
+    sudo systemctl stop systemd-timesyncd
+    sudo systemctl start systemd-timesyncd
+    date +"Time reset to NTP"
+    echo "Restarting cron"
+    sudo systemctl restart cron
+    ;;
+
+  *)
+    echo "Usage: $0 {current_time|end_of_month|end_of_day <days_offset>|reset_time}"
+    exit 1
+    ;;
+esac

core/component/base/event_handler.py

@@ -5,6 +5,7 @@ from django.utils import timezone
 
 from core.models import Invoice, BillingProject
 from core.component.base.invoice_handler import InvoiceHandler
+from core.utils.date_utils import current_localtime
 
 
 class EventHandler(metaclass=abc.ABCMeta):
@@ -22,7 +23,7 @@ class EventHandler(metaclass=abc.ABCMeta):
         invoice = Invoice.objects.filter(project__tenant_id=tenant_id, state=Invoice.InvoiceState.IN_PROGRESS).first()
         if not invoice:
             project = BillingProject.objects.get_or_create(tenant_id=tenant_id)
-            date_today = timezone.now()
+            date_today = current_localtime()
             month_first_day = date_today.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
             invoice = Invoice.objects.create(
                 project=project,
@@ -68,7 +69,7 @@ class EventHandler(metaclass=abc.ABCMeta):
         instance = self.invoice_handler.get_active_instance(invoice, payload)
         if not instance:
             payload['invoice'] = invoice
-            payload['start_date'] = timezone.now()
+            payload['start_date'] = current_localtime()
 
             self.invoice_handler.create(payload, fallback_price=True)
 
@@ -113,7 +114,7 @@ class EventHandler(metaclass=abc.ABCMeta):
 
         if instance:
             if self.invoice_handler.is_price_dependency_changed(instance, payload):
-                self.invoice_handler.roll(instance, close_date=timezone.now(), update_payload=payload, fallback_price=True)
+                self.invoice_handler.roll(instance, close_date=current_localtime(), update_payload=payload, fallback_price=True)
                 return True
 
             if self.invoice_handler.is_informative_changed(instance, payload):

core/component/base/invoice_handler.py

@@ -7,6 +7,7 @@ from djmoney.money import Money
 from core.exception import PriceNotFound
 from core.models import InvoiceComponentMixin, PriceMixin
 from core.notification import send_notification
+from core.utils.date_utils import current_localtime
 
 
 class InvoiceHandler(metaclass=abc.ABCMeta):
@@ -130,7 +131,7 @@ class InvoiceHandler(metaclass=abc.ABCMeta):
         :return:
         """
         self.update(instance, payload, save=False)
-        instance.close(timezone.now())  # Close will also save the instance
+        instance.close(current_localtime())  # Close will also save the instance
 
     def is_informative_changed(self, instance, payload):
         """

core/component/project/event_handler.py

@@ -3,6 +3,8 @@ import logging
 from core.models import BillingProject, Invoice
 from django.utils import timezone
 
+from core.utils.date_utils import current_localtime
+
 LOG = logging.getLogger("yuyu_notification")
 
 
@@ -18,7 +20,7 @@ class ProjectEventHandler:
             self.init_first_invoice(project)
 
     def init_first_invoice(self, project):
-        date_today = timezone.now()
+        date_today = current_localtime()
         month_first_day = date_today.replace(day=1, hour=0, minute=0, second=0, microsecond=0)
         Invoice.objects.create(
             project=project,

core/event_endpoint.py

@@ -5,6 +5,7 @@ from oslo_messaging import NotificationResult
 
 from core.component import component
 from core.component.project.event_handler import ProjectEventHandler
+from core.feature.unpaid_invoice_handle.event_handler import UnpaidInvoiceEventHandler
 from core.notification import send_notification
 from core.utils.dynamic_setting import get_dynamic_settings, BILLING_ENABLED, get_dynamic_setting
 from yuyu import settings
@@ -20,6 +21,7 @@ class EventEndpoint(object):
 
         # Add handler for project event
         self.event_handler.append(ProjectEventHandler())
+        self.event_handler.append(UnpaidInvoiceEventHandler())
 
     def info(self, ctxt, publisher_id, event_type, payload, metadata):
         LOG.info("=== Event Received ===")

core/feature/unpaid_invoice_handle/actions.py

@@ -0,0 +1,9 @@
+from enum import Enum
+
+
+class UnpainInvoiceAction(Enum):
+    SEND_MESSAGE = "send_message"
+    STOP_INSTANCE = "stop_instance"
+    SUSPEND_INSTANCE = "suspend_instance"
+    PAUSE_INSTANCE = "pause_instance"
+    DELETE_INSTANCE = "delete_instance"

core/feature/unpaid_invoice_handle/command.py

@@ -0,0 +1,146 @@
+import logging
+import traceback
+
+import openstack
+from django.utils import timezone
+from core.feature.unpaid_invoice_handle.actions import UnpainInvoiceAction
+
+from core.models import Invoice
+from core.notification import send_notification
+from core.utils.date_utils import current_localtime
+from core.utils.dynamic_setting import get_dynamic_setting, BILLING_ENABLED
+from yuyu import settings
+
+
+LOG = logging.getLogger("yuyu")
+
+class UnpaidInvoiceHandlerCommand:
+    def handle(self):
+        LOG.info("Processing Unpaid Invoice")
+
+        # Initialize connection
+        if not get_dynamic_setting(BILLING_ENABLED):
+            return
+        
+        if not hasattr(settings, 'UNPAID_INVOICE_HANDLER_CONFIG'):
+            LOG.exception("Missing UNPAID_INVOICE_HANDLER_CONFIG in settings")
+            return
+
+        expired_unpaid_invoices = Invoice.objects.filter(state=Invoice.InvoiceState.UNPAID).all()
+        for invoice in expired_unpaid_invoices:
+            self.run_action_on_config(invoice)
+                
+        LOG.info("Processing Unpaid Invoice Done")
+    
+    def run_action_on_config(self, invoice):
+        schedule_config = settings.UNPAID_INVOICE_HANDLER_CONFIG
+        date_diff = current_localtime() - invoice.end_date
+        past_day = date_diff.days
+        LOG.info(f"Processing Unpaid Invoice {invoice.id} with past day {past_day}")
+        for config in schedule_config:
+            if config['day'] == past_day:
+                LOG.info('Running action')
+                self.run_action(invoice, config['action'], config)
+
+    def run_action(self, invoice, action, config=None):
+        LOG.info(f"Running action {action} with config {config}")
+        try:
+            if action == UnpainInvoiceAction.SEND_MESSAGE.value:
+                self._send_message(invoice, config)
+
+            if action == UnpainInvoiceAction.STOP_INSTANCE.value:
+                self._stop_component(invoice)
+
+            if action == UnpainInvoiceAction.SUSPEND_INSTANCE.value:
+                self._stop_component(invoice)
+
+            if action == UnpainInvoiceAction.PAUSE_INSTANCE.value:
+                self._stop_component(invoice)
+
+            if action == UnpainInvoiceAction.DELETE_INSTANCE.value:
+                self._delete_component(invoice)
+        except Exception:
+            LOG.exception("Failed to process Unpaid Invoice")
+            send_notification(
+                project=None,
+                title='[Error] Error when processing unpaid invoice',
+                short_description=f'There is an error when processing unpaid invoice',
+                content=f'There is an error when handling unpaid invoice \n {traceback.format_exc()}',
+            )
+    
+    def _stop_component(self, invoice: Invoice):
+        LOG.info("Stopping Component")
+        conn = openstack.connect(cloud=settings.CLOUD_CONFIG_NAME)
+
+        # Stop Instance
+        for server in conn.compute.servers(all_projects=True, project_id=invoice.project.tenant_id):
+            LOG.info(f"Stopping server {server.id}")
+            conn.compute.stop_server(server)
+
+    def _suspend_component(self, invoice: Invoice):
+        LOG.info("Suspending Component")
+        conn = openstack.connect(cloud=settings.CLOUD_CONFIG_NAME)
+
+        # Suspend Instance
+        for server in conn.compute.servers(all_projects=True, project_id=invoice.project.tenant_id):
+            LOG.info(f"Suspending server {server.id}")
+            
+            conn.compute.suspend_server(server)
+
+    def _pause_component(self, invoice: Invoice):
+        LOG.info("Pausing Component")
+        conn = openstack.connect(cloud=settings.CLOUD_CONFIG_NAME)
+
+        # Pause Instance
+        for server in conn.compute.servers(all_projects=True, project_id=invoice.project.tenant_id):
+            LOG.info(f"Pausing server {server.id}")
+            
+            conn.compute.pause_server(server)
+
+    def _delete_component(self, invoice: Invoice):
+        LOG.info("Deleting Component")
+        conn = openstack.connect(cloud=settings.CLOUD_CONFIG_NAME)
+
+        # Delete Instance
+        for server in conn.compute.servers(all_projects=True, project_id=invoice.project.tenant_id):
+            LOG.info(f"Deleting server {server.id}")
+            
+            conn.compute.delete_server(server)
+
+        # Delete Image
+        for image in conn.compute.images(all_projects=True, project_id=invoice.project.tenant_id):
+            LOG.info(f"Deleting image {image.id}")
+            conn.compute.delete_image(image)
+
+        # Delete Floating Ips
+        for ip in conn.network.ips(all_projects=True, project_id=invoice.project.tenant_id):
+            LOG.info(f"Deleting ip {ip.id}")
+            
+            conn.network.delete_ip(ip)
+
+        # Delete Router
+        for route in conn.network.routers(all_projects=True, project_id=invoice.project.tenant_id):
+            LOG.info(f"Deleting route {route.id}")
+            
+            conn.network.delete_router(route)
+
+        # Delete Volume
+        for volume in conn.block_storage.volumes(all_projects=True, project_id=invoice.project.tenant_id):
+            LOG.info(f"Deleting volume {volume.id}")
+            
+            conn.block_storage.delete_volume(volume)
+
+        # Delete Snapshot
+        for snapshot in conn.block_storage.snapshots(all_projects=True, project_id=invoice.project.tenant_id):
+            LOG.info(f"Deleting snapshot {snapshot.id}")
+            
+            conn.block_storage.delete_snapshot(snapshot)
+
+    def _send_message(self, invoice: Invoice, message_config):
+        LOG.info("Sending Message")
+        send_notification(
+            project=invoice.project,
+            title=message_config['message_title'],
+            short_description=message_config['message_short_description'],
+            content=message_config['message_content'],
+        )

core/feature/unpaid_invoice_handle/event_handler.py

@@ -0,0 +1,70 @@
+from datetime import timezone
+import logging
+from core.feature.unpaid_invoice_handle.actions import UnpainInvoiceAction
+from core.feature.unpaid_invoice_handle.command import UnpaidInvoiceHandlerCommand
+from core.models import Invoice
+from core.utils.date_utils import current_localtime
+from yuyu import settings
+
+LOG = logging.getLogger("yuyu")
+
+class UnpaidInvoiceEventHandler:
+    filter_action = [
+        UnpainInvoiceAction.STOP_INSTANCE.value,
+        UnpainInvoiceAction.SUSPEND_INSTANCE.value,
+        UnpainInvoiceAction.PAUSE_INSTANCE.value,
+        UnpainInvoiceAction.DELETE_INSTANCE.value,
+    ]
+
+    def filter_event_project_id(event_type, raw_payload):
+        if event_type == 'floatingip.create.end':
+            return raw_payload['floatingip']['tenant_id']
+        if event_type == 'image.activate':
+            return raw_payload['owner']
+        if event_type == 'compute.instance.update':
+            return raw_payload['tenant_id']
+        if event_type == 'router.create.end':
+            return raw_payload['router']['tenant_id']
+        if event_type == 'router.update.end':
+            return raw_payload['router']['tenant_id']
+        if event_type == 'snapshot.create.end':
+            return raw_payload['tenant_id']
+        if event_type == 'volume.create.end':
+            return raw_payload['tenant_id']
+
+        return None
+    
+    def handle(self, event_type, raw_payload):
+        try:
+            LOG.exception("Processing Unpaid Invoice Event")
+            
+            if not hasattr(settings, 'UNPAID_INVOICE_HANDLER_CONFIG'):
+                LOG.exception("Missing UNPAID_INVOICE_HANDLER_CONFIG in settings")
+                return
+            
+            schedule_config = settings.UNPAID_INVOICE_HANDLER_CONFIG
+            command = UnpaidInvoiceHandlerCommand()
+            project_id = self.filter_event_project_id(event_type, raw_payload)
+            if project_id:
+                # Fetch All Unpaid Invoice
+                unpaid_invoice = Invoice.objects.filter(project__tenant_id=project_id, state=Invoice.InvoiceState.UNPAID).first()
+                used_config = {}
+                used_day = 0
+
+                if unpaid_invoice:
+                    # Calculate Days
+                    # Find last config that has beed runned in past days
+                    date_diff = current_localtime() - unpaid_invoice.end_date
+                    past_day = date_diff.days
+                    for config in schedule_config:
+                        if config['action'] not in self.filter_action:
+                            continue
+
+                        if config['day'] <= past_day and config['day'] > used_day:
+                            used_day = config['day']
+                            used_config = config
+
+                    if used_config:
+                        command.run_action(unpaid_invoice, used_config['action'], config)
+        except Exception:
+            LOG.exception("Failed to process Unpaid Invoice Event")

core/management/commands/check_openstack_connection.py

@@ -0,0 +1,20 @@
+import logging
+
+import openstack
+from django.core.management import BaseCommand
+
+from yuyu import settings
+
+LOG = logging.getLogger("check_openstack_connection")
+
+
+class Command(BaseCommand):
+    help = 'Yuyu Check Openstack Connection'
+
+    def handle(self, *args, **options):
+        print("Checking Openstack Connection")
+        print("Will try to list instance on the project that configured in `clouds.yaml`")
+
+        conn = openstack.connect(cloud=settings.CLOUD_CONFIG_NAME)
+        for server in conn.compute.servers(all_projects=True):
+            print(server.name)

core/management/commands/handle_unpaid_invoice.py

@@ -0,0 +1,21 @@
+import logging
+import traceback
+
+import openstack
+from django.core.management import BaseCommand
+from django.utils import timezone
+from core.feature.unpaid_invoice_handle.command import UnpaidInvoiceHandlerCommand
+
+from core.models import Invoice
+from core.notification import send_notification
+from core.utils.dynamic_setting import get_dynamic_setting, BILLING_ENABLED
+from yuyu import settings
+
+LOG = logging.getLogger("yuyu")
+
+
+class Command(BaseCommand):
+    unpaid_invoice_command = UnpaidInvoiceHandlerCommand()
+    
+    def handle(self, *args, **options):
+        self.unpaid_invoice_command.handle()

core/management/commands/process_invoice.py

@@ -8,6 +8,7 @@ from django.utils import timezone
 from core.component import component, labels
 from core.models import Invoice, InvoiceComponentMixin, Balance
 from core.notification import send_notification_from_template, send_notification
+from core.utils.date_utils import current_localtime
 from core.utils.dynamic_setting import get_dynamic_setting, BILLING_ENABLED, INVOICE_TAX, COMPANY_NAME, \
     COMPANY_ADDRESS, INVOICE_AUTO_DEDUCT_BALANCE
 from yuyu import settings
@@ -25,8 +26,11 @@ class Command(BaseCommand):
                 LOG.info("Billing not activated")
                 return
 
-            self.close_date = timezone.now()
+            self.close_date = current_localtime()
+            LOG.info(f"Close Date: {self.close_date}")
+            
             self.tax_pertentage = get_dynamic_setting(INVOICE_TAX)
+            LOG.info(f"Tax Percentage: {self.tax_pertentage}")
 
             active_invoices = Invoice.objects.filter(state=Invoice.InvoiceState.IN_PROGRESS).all()
             for active_invoice in active_invoices:

core/models.py

@@ -13,6 +13,7 @@ from djmoney.money import Money
 from core.component import labels
 from core.component.labels import LABEL_INSTANCES, LABEL_IMAGES, LABEL_SNAPSHOTS, LABEL_ROUTERS, LABEL_FLOATING_IPS, \
     LABEL_VOLUMES
+from core.utils.date_utils import DateTimeLocalField, current_localtime
 from core.utils.model_utils import BaseModel, TimestampMixin, PriceMixin, InvoiceComponentMixin
 
 LOG = logging.getLogger("yuyu")
@@ -80,9 +81,9 @@ class Invoice(BaseModel, TimestampMixin):
         FINISHED = 100
 
     project = models.ForeignKey('BillingProject', on_delete=models.CASCADE)
-    start_date = models.DateTimeField()
+    start_date = DateTimeLocalField()
-    end_date = models.DateTimeField(default=None, blank=True, null=True)
+    end_date = DateTimeLocalField(default=None, blank=True, null=True)
-    finish_date = models.DateTimeField(default=None, blank=True, null=True)
+    finish_date = DateTimeLocalField(default=None, blank=True, null=True)
     state = models.IntegerField(choices=InvoiceState.choices)
     tax = MoneyField(max_digits=256, default=None, blank=True, null=True)
     total = MoneyField(max_digits=256, default=None, blank=True, null=True)
@@ -127,12 +128,11 @@ class Invoice(BaseModel, TimestampMixin):
         self.end_date = date
         self.tax = tax_percentage * self.subtotal / 100
         self.total = self.tax + self.subtotal
-        # TODO: Deduct balance
         self.save()
 
     def finish(self):
         self.state = Invoice.InvoiceState.FINISHED
-        self.finish_date = timezone.now()
+        self.finish_date = current_localtime()
         self.save()
 
     def rollback_to_unpaid(self):

core/utils/date_utils.py

@@ -0,0 +1,15 @@
+from typing import Any
+from django.utils import timezone
+from django.db import models
+
+def current_localtime():
+    return timezone.localtime(timezone.now())
+
+# A DateTimeField that return local time instead of UTC to make it easier to calculate duration
+class DateTimeLocalField(models.DateTimeField):
+    def from_db_value(self, value, expression, connection) -> Any:
+        if value is None:
+            return None
+        else:
+            return timezone.localtime(value)
+    

core/utils/model_utils.py

@@ -5,6 +5,8 @@ from django.utils import timezone
 from djmoney.models.fields import MoneyField
 from django.utils.timesince import timesince
 
+from core.utils.date_utils import DateTimeLocalField, current_localtime
+
 
 class BaseModel(models.Model):
     class Meta:
@@ -31,13 +33,13 @@ class InvoiceComponentMixin(TimestampMixin, PriceMixin):
     """
     Storing start time for price calculation
     """
-    start_date = models.DateTimeField()
+    start_date = DateTimeLocalField()
 
     """
     Storing end time of the component, when component still active it will be None.
     It will be set when component is closed or rolled
     """
-    end_date = models.DateTimeField(default=None, blank=True, null=True)
+    end_date = DateTimeLocalField(default=None, blank=True, null=True)
 
     @property
     def adjusted_end_date(self):
@@ -46,7 +48,7 @@ class InvoiceComponentMixin(TimestampMixin, PriceMixin):
         Basically it just return current time if end_date is None
         end_date will be set when invoice is finished every end of the month or when invoice component is rolled
         """
-        current_date = timezone.now()
+        current_date = current_localtime()
         if self.end_date:
             end_date = self.end_date
         else:
@@ -84,7 +86,7 @@ class InvoiceComponentMixin(TimestampMixin, PriceMixin):
 
     def close(self, date):
         """
-        Close component the component
+        Close the component
         """
         self.end_date = date
         self.save()

requirements.txt

@@ -4,7 +4,7 @@ pytz==2021.1
 sqlparse==0.4.1
 oslo.messaging>=12.8.0
 django-money==2.0.1
-openstacksdk==0.58.0
+openstacksdk==2.0.0
 djangorestframework==3.12.4
 django-filter==2.4.0
 Markdown==3.3.4

yuyu/local_settings.py.sample

@@ -15,4 +15,69 @@ EMAIL_HOST = 'smtp.gmail.com'
 EMAIL_HOST_USER = '<paste your gmail account here>'
 EMAIL_HOST_PASSWORD = '<paste Google password or app password here>'
 EMAIL_PORT = 587
-EMAIL_USE_TLS = True
+EMAIL_USE_TLS = True
+
+"""
+Unpaid Invoice Handler Configuration
+
+CLOUD_CONFIG_NAME : Is configuration in clouds.yaml that you want to use to connect to openstack.
+Example: Your clouds.yaml is
+clouds:
+  openstack:
+    auth:
+      auth_url: http://172.10.10.150:5000
+      username: "admin"
+      password: "your_password"
+      project_id: 0000000000000000000000000
+      project_name: "admin"
+      user_domain_name: "Default"
+    region_name: "ID"
+    interface: "public"
+    identity_api_version: 3
+
+You can put `openstack` as CLOUD_CONFIG_NAME
+
+UNPAID_INVOICE_HANDLER_CONFIG : Is configuration for an action that will be run on a particular day after invoice is issued and still unpaid
+The format is a list of dictionary that follow
+{
+   "day": 0, # The day of the action that will be run, example if you put 5 it will run in 5 day after invoice is issued and still unpaid
+   "action": "send_message/stop_instance/delete_instance", # The action that you want to run
+   "message_title": "Title", # The title or subject of the message if you use `send_message` action
+   "message_short_description": "Short Description", # The short description of message if you use `send_message` action
+   "message_content": "Title", # The content of the message if you use `send_message` action
+}
+"""
+
+#
+# CLOUD_CONFIG_NAME = "openstack"
+# UNPAID_INVOICE_HANDLER_CONFIG = [
+#     {
+#         "day": 5,
+#         "action": "send_message",
+#         "message_title": "Your invoice has been expired. Please pay now!",
+#         "message_short_description": "Your invoice has been expired. Please pay now!",
+#         "message_content": "Your invoice has been expired. Please pay now!",
+#     },
+#     {
+#         "day": 10,
+#         "action": "stop_instance",
+#     },
+#     {
+#         "day": 10,
+#         "action": "send_message",
+#         "message_title": "Your compute instance will be stopped",
+#         "message_short_description": "Your compute instance will be stopped",
+#         "message_content": "Your compute instance will be stopped because you have unpaid invoice",
+#     },
+#     {
+#         "day": 15,
+#         "action": "send_message",
+#         "message_title": "All of your instance has been deleted",
+#         "message_short_description": "All of your instance has been deleted",
+#         "message_content": "All of your instance has been deleted because you have unpaid invoice",
+#     },
+#     {
+#         "day": 15,
+#         "action": "delete_instance",
+#     },
+# ]